2023-09-28 19:53:14,620 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-28 19:53:14,943 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-28 19:53:15,056 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-28 19:53:15,120 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-28 19:53:15,128 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-28 19:53:15,144 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=171, AvailableMemoryMB=6568
2023-09-28 19:53:15,146 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-28 19:53:15,146 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-28 19:53:15,287 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv104146462:84 , srv1045008004:84 , srv1096889474:84 , srv1128052962:84 , srv1185341860:84 , srv1204121728:84 , srv1215051503:84 , srv1222734448:84 , srv1242937310:84 , srv1285092984:84 , srv1339168364:84 , srv1343562138:84 , srv1363598059:84 , srv1382608998:84 , srv1384462473:84 , srv139760886:84 , srv1440370998:84 , srv1453046460:84 , srv1454991571:84 , srv1495494503:84 , srv1563154569:84 , srv1577415130:84 , srv1577439769:84 , srv1585716862:84 , srv1610455955:84 , srv1633689953:84 , srv1693472033:84 , srv1694253782:84 , srv1719034044:84 , srv1777456876:84 , srv1793877172:84 , srv1800981679:84 , srv1812551542:84 , srv1849511341:84 , srv1854968203:84 , srv1857520447:84 , srv1862962542:84 , srv1870476916:84 , srv1908907265:84 , srv1920593791:84 , srv1925473384:84 , srv2008410997:84 , srv2018916427:84 , srv2026171542:84 , srv2057159842:84 , srv2075941256:84 , srv2079195677:84 , srv2112592896:84 , srv2127988923:84 , srv2132192012:84 , srv2146364398:84 , srv217971693:84 , srv220673589:84 , srv262367094:84 , srv292380380:84 , srv297555996:84 , srv310097732:84 , srv319367091:84 , srv36637626:84 , srv38667306:84 , srv389043537:84 , srv389594742:84 , srv400193896:84 , srv415478630:84 , srv428835040:84 , srv45235417:84 , srv464399430:84 , srv467077695:84 , srv467547054:84 , srv47174726:84 , srv47963581:84 , srv489169636:84 , srv508954217:84 , srv509515776:84 , srv535155077:84 , srv568231242:84 , srv598064510:84 , srv610376741:84 , srv617689389:84 , srv62040577:84 , srv624352859:84 , srv636223582:84 , srv669376881:84 , srv671616411:84 , srv675694925:84 , srv707292966:84 , srv726882459:84 , srv735854960:84 , srv738340361:84 , srv76230643:84 , srv78563019:84 , srv812226899:84 , srv829686954:84 , srv869951816:84 , srv874863077:84 , srv908746042:84 , srv910626894:84 , srv958489911:84 , srv985159858:84 , srv1821423719:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-28 19:53:15,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:53:15,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2146364398=51, srv262367094=54, srv874863077=95, srv535155077=75, srv389594742=62, srv217971693=52, srv62040577=80, srv675694925=85, srv1563154569=20, srv415478630=64, srv910626894=97, srv624352859=81, srv908746042=96, srv2075941256=46, srv2127988923=49, srv389043537=61, srv400193896=63, srv1128052962=3, srv735854960=88, srv47963581=71, srv610376741=78, srv568231242=76, srv1339168364=10, srv598064510=77, srv1920593791=40, srv671616411=84, srv1694253782=27, srv78563019=91, srv1440370998=16, srv1045008004=1, srv310097732=57, srv2079195677=47, srv319367091=58, srv1096889474=2, srv38667306=60, srv1215051503=6, srv1719034044=28, srv2026171542=44, srv1382608998=13, srv1585716862=23, srv1610455955=24, srv509515776=74, srv1821423719=33, srv1870476916=38, srv76230643=90, srv1285092984=9, srv1849511341=34, srv1793877172=30, srv428835040=65, srv1577439769=22, srv467077695=68, srv829686954=93, srv1363598059=12, srv47174726=70, srv1862962542=37, srv1384462473=14, srv812226899=92, srv104146462=0, srv464399430=67, srv669376881=83, srv707292966=86, srv1908907265=39, srv1495494503=19, srv1693472033=26, srv726882459=87, srv1343562138=11, srv467547054=69, srv220673589=53, srv292380380=55, srv738340361=89, srv1242937310=8, srv139760886=15, srv1453046460=17, srv2008410997=42, srv1577415130=21, srv36637626=59, srv869951816=94, srv2112592896=48, srv985159858=99, srv1633689953=25, srv1204121728=5, srv489169636=72, srv636223582=82, srv2057159842=45, srv508954217=73, srv1857520447=36, srv2018916427=43, srv2132192012=50, srv1777456876=29, srv45235417=66, srv1185341860=4, srv958489911=98, srv1925473384=41, srv1812551542=32, srv1454991571=18, srv297555996=56, srv1854968203=35, srv617689389=79, srv1800981679=31, srv1222734448=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:53:15,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:53:15,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:53:15,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:53:15,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:53:15,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:53:15,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:53:15,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:53:15,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:53:15,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:53:15,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:53:15,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:53:15,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:53:15,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:53:15,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:53:15,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:53:15,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:53:15,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:53:15,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:53:15,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 19:53:15,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 19:53:15,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 19:53:15,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 19:53:15,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 19:53:15,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 19:53:15,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 19:53:15,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 19:53:15,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 19:53:15,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 19:53:15,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 19:53:15,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 19:53:15,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 19:53:15,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 19:53:15,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 19:53:15,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 19:53:15,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 19:53:15,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 19:53:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 19:53:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 19:53:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 19:53:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 19:53:15,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 19:53:15,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 19:53:15,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 19:53:15,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 19:53:15,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 19:53:15,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 19:53:15,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 19:53:15,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 19:53:15,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:15,590 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 19:53:15,590 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 19:53:15,998 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 560 ms to try 5000 different iterations.  Found a solution that moves 4669 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18428032924130477. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040287655044952346, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06157491581916231, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5187777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040287655044952346, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17958333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:16,011 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1242937310:82 , srv62040577:82 , srv1045008004:83 , srv1185341860:83 , srv1204121728:83 , srv1577415130:83 , srv1610455955:83 , srv2008410997:83 , srv2057159842:83 , srv217971693:83 , srv45235417:83 , srv464399430:83 , srv489169636:83 , srv508954217:83 , srv669376881:83 , srv707292966:83 , srv908746042:83 , srv1128052962:84 , srv1453046460:84 , srv1693472033:84 , srv1857520447:84 , srv1920593791:84 , srv2018916427:84 , srv297555996:84 , srv319367091:84 , srv76230643:84 , srv812226899:84 , srv1215051503:85 , srv1363598059:85 , srv1719034044:85 , srv1800981679:85 , srv1812551542:85 , srv1870476916:85 , srv2075941256:85 , srv2079195677:85 , srv2127988923:85 , srv2146364398:85 , srv220673589:85 , srv36637626:85 , srv467077695:85 , srv617689389:85 , srv671616411:85 , srv78563019:85 , srv1096889474:86 , srv1382608998:86 , srv1694253782:86 , srv1793877172:86 , srv389594742:86 , srv467547054:86 , srv726882459:86 , srv829686954:86 , srv1339168364:87 , srv1454991571:87 , srv1633689953:87 , srv1849511341:87 , srv1862962542:87 , srv1925473384:87 , srv2026171542:87 , srv2112592896:87 , srv38667306:87 , srv415478630:87 , srv428835040:87 , srv624352859:87 , srv636223582:87 , srv738340361:87 , srv869951816:87 , srv985159858:87 , srv1222734448:88 , srv1585716862:88 , srv1908907265:88 , srv47174726:88 , srv509515776:88 , srv568231242:88 , srv598064510:88 , srv610376741:88 , srv675694925:88 , srv735854960:88 , srv874863077:88 , srv910626894:88 , srv958489911:88 , srv104146462:89 , srv1440370998:89 , srv1563154569:89 , srv262367094:89 , srv292380380:89 , srv400193896:89 , srv47963581:89 , srv1343562138:90 , srv1384462473:90 , srv2132192012:90 , srv139760886:91 , srv1777456876:91 , srv310097732:91 , srv389043537:91 , srv535155077:91 , srv1495494503:92 , srv1854968203:92 , srv1577439769:93 , srv1285092984:94 , srv1821423719:448 }
2023-09-28 19:53:16,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:53:16,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2146364398=51, srv262367094=54, srv874863077=95, srv535155077=75, srv389594742=62, srv217971693=52, srv62040577=80, srv675694925=85, srv1563154569=20, srv415478630=64, srv910626894=97, srv624352859=81, srv908746042=96, srv2075941256=46, srv2127988923=49, srv389043537=61, srv400193896=63, srv1128052962=3, srv735854960=88, srv47963581=71, srv610376741=78, srv568231242=76, srv1339168364=10, srv598064510=77, srv1920593791=40, srv671616411=84, srv1694253782=27, srv78563019=91, srv1440370998=16, srv1045008004=1, srv310097732=57, srv2079195677=47, srv319367091=58, srv1096889474=2, srv38667306=60, srv1215051503=6, srv1719034044=28, srv2026171542=44, srv1382608998=13, srv1585716862=23, srv1610455955=24, srv509515776=74, srv1821423719=33, srv1870476916=38, srv76230643=90, srv1285092984=9, srv1849511341=34, srv1793877172=30, srv428835040=65, srv1577439769=22, srv467077695=68, srv829686954=93, srv1363598059=12, srv47174726=70, srv1862962542=37, srv1384462473=14, srv812226899=92, srv104146462=0, srv464399430=67, srv669376881=83, srv707292966=86, srv1908907265=39, srv1495494503=19, srv1693472033=26, srv726882459=87, srv1343562138=11, srv467547054=69, srv220673589=53, srv292380380=55, srv738340361=89, srv1242937310=8, srv139760886=15, srv1453046460=17, srv2008410997=42, srv1577415130=21, srv36637626=59, srv869951816=94, srv2112592896=48, srv985159858=99, srv1633689953=25, srv1204121728=5, srv489169636=72, srv636223582=82, srv2057159842=45, srv508954217=73, srv1857520447=36, srv2018916427=43, srv2132192012=50, srv1777456876=29, srv45235417=66, srv1185341860=4, srv958489911=98, srv1925473384=41, srv1812551542=32, srv1454991571=18, srv297555996=56, srv1854968203=35, srv617689389=79, srv1800981679=31, srv1222734448=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:53:16,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:53:16,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 19:53:16,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 19:53:16,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 19:53:16,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 19:53:16,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 19:53:16,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 19:53:16,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 19:53:16,201 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040287655044952346, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06157491581916231, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040287655044952346, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17958333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:16,202 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 19:53:16,202 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18424763620454251, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040287655044952346, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06157491581916231, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040287655044952346, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17958333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2455, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 19:53:16,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 531 ms to try 5000 different iterations.  Found a solution that moves 2613 regions; Going from a computed imbalance of 0.18424763620454251 to a new imbalance of 0.015887420997765574. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022050401968240117); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04012886089826303, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29033333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022050401968240117); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:16,608 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1285092984:78 , srv1343562138:79 , srv1495494503:79 , srv1577439769:79 , srv139760886:80 , srv1563154569:80 , srv1854968203:80 , srv1222734448:81 , srv262367094:81 , srv310097732:81 , srv389043537:81 , srv1384462473:82 , srv1440370998:82 , srv1777456876:82 , srv1908907265:82 , srv292380380:82 , srv610376741:82 , srv104146462:83 , srv2132192012:83 , srv400193896:83 , srv47963581:83 , srv535155077:83 , srv568231242:83 , srv675694925:83 , srv735854960:83 , srv738340361:83 , srv985159858:83 , srv1339168364:84 , srv1585716862:84 , srv1633689953:84 , srv1793877172:84 , srv1849511341:84 , srv1925473384:84 , srv2112592896:84 , srv829686954:84 , srv910626894:84 , srv958489911:84 , srv1454991571:85 , srv1694253782:85 , srv1719034044:85 , srv2026171542:85 , srv220673589:85 , srv415478630:85 , srv428835040:85 , srv47174726:85 , srv624352859:85 , srv636223582:85 , srv874863077:85 , srv1920593791:86 , srv509515776:86 , srv598064510:86 , srv617689389:86 , srv726882459:86 , srv78563019:86 , srv1096889474:87 , srv1185341860:87 , srv1215051503:87 , srv1382608998:87 , srv2018916427:87 , srv2075941256:87 , srv2079195677:87 , srv2146364398:87 , srv297555996:87 , srv812226899:87 , srv869951816:87 , srv1363598059:88 , srv1693472033:88 , srv1800981679:88 , srv1812551542:88 , srv1857520447:88 , srv2127988923:88 , srv319367091:88 , srv36637626:88 , srv38667306:88 , srv389594742:88 , srv467077695:88 , srv671616411:88 , srv1204121728:89 , srv1453046460:89 , srv1577415130:89 , srv1862962542:89 , srv1870476916:89 , srv2008410997:89 , srv2057159842:89 , srv217971693:89 , srv467547054:89 , srv669376881:89 , srv76230643:89 , srv1045008004:90 , srv1128052962:90 , srv1242937310:90 , srv1610455955:90 , srv45235417:90 , srv464399430:90 , srv508954217:90 , srv707292966:90 , srv62040577:91 , srv908746042:91 , srv489169636:92 , srv1821423719:522 }
2023-09-28 19:53:16,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:53:16,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2146364398=51, srv262367094=54, srv874863077=95, srv535155077=75, srv389594742=62, srv217971693=52, srv62040577=80, srv675694925=85, srv1563154569=20, srv415478630=64, srv910626894=97, srv624352859=81, srv908746042=96, srv2075941256=46, srv2127988923=49, srv389043537=61, srv400193896=63, srv1128052962=3, srv735854960=88, srv47963581=71, srv610376741=78, srv568231242=76, srv1339168364=10, srv598064510=77, srv1920593791=40, srv671616411=84, srv1694253782=27, srv78563019=91, srv1440370998=16, srv1045008004=1, srv310097732=57, srv2079195677=47, srv319367091=58, srv1096889474=2, srv38667306=60, srv1215051503=6, srv1719034044=28, srv2026171542=44, srv1382608998=13, srv1585716862=23, srv1610455955=24, srv509515776=74, srv1821423719=33, srv1870476916=38, srv76230643=90, srv1285092984=9, srv1849511341=34, srv1793877172=30, srv428835040=65, srv1577439769=22, srv467077695=68, srv829686954=93, srv1363598059=12, srv47174726=70, srv1862962542=37, srv1384462473=14, srv812226899=92, srv104146462=0, srv464399430=67, srv669376881=83, srv707292966=86, srv1908907265=39, srv1495494503=19, srv1693472033=26, srv726882459=87, srv1343562138=11, srv467547054=69, srv220673589=53, srv292380380=55, srv738340361=89, srv1242937310=8, srv139760886=15, srv1453046460=17, srv2008410997=42, srv1577415130=21, srv36637626=59, srv869951816=94, srv2112592896=48, srv985159858=99, srv1633689953=25, srv1204121728=5, srv489169636=72, srv636223582=82, srv2057159842=45, srv508954217=73, srv1857520447=36, srv2018916427=43, srv2132192012=50, srv1777456876=29, srv45235417=66, srv1185341860=4, srv958489911=98, srv1925473384=41, srv1812551542=32, srv1454991571=18, srv297555996=56, srv1854968203=35, srv617689389=79, srv1800981679=31, srv1222734448=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:53:16,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:53:16,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 19:53:16,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 19:53:16,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 19:53:16,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 19:53:16,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 19:53:16,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 19:53:16,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 19:53:16,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 19:53:16,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 19:53:16,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 19:53:16,786 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022050401968240117); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04012886089826303, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022050401968240117); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:16,786 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 19:53:16,786 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015869124380704137, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022050401968240117); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04012886089826303, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022050401968240117); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012833333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04475, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 19:53:17,136 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 461 ms to try 5000 different iterations.  Found a solution that moves 981 regions; Going from a computed imbalance of 0.015869124380704137 to a new imbalance of 7.081063441642522E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006217570812329759); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012463189345605802); MoveCostFunction : (multiplier=7.0, imbalance=0.109, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006217570812329759); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:17,139 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1862962542:81 , srv1870476916:82 , srv467547054:82 , srv489169636:82 , srv1128052962:83 , srv38667306:83 , srv389594742:83 , srv598064510:83 , srv869951816:83 , srv1096889474:84 , srv1204121728:84 , srv1363598059:84 , srv1382608998:84 , srv1453046460:84 , srv1577439769:84 , srv1610455955:84 , srv2127988923:84 , srv36637626:84 , srv467077695:84 , srv508954217:84 , srv509515776:84 , srv535155077:84 , srv624352859:84 , srv874863077:84 , srv908746042:84 , srv1045008004:85 , srv1215051503:85 , srv1285092984:85 , srv1384462473:85 , srv1454991571:85 , srv1585716862:85 , srv1777456876:85 , srv1800981679:85 , srv1812551542:85 , srv1854968203:85 , srv2026171542:85 , srv2057159842:85 , srv2132192012:85 , srv2146364398:85 , srv292380380:85 , srv389043537:85 , srv415478630:85 , srv464399430:85 , srv47174726:85 , srv47963581:85 , srv62040577:85 , srv669376881:85 , srv671616411:85 , srv707292966:85 , srv76230643:85 , srv910626894:85 , srv104146462:86 , srv1185341860:86 , srv1242937310:86 , srv139760886:86 , srv1440370998:86 , srv1495494503:86 , srv1633689953:86 , srv1693472033:86 , srv1694253782:86 , srv1857520447:86 , srv2008410997:86 , srv2018916427:86 , srv2075941256:86 , srv2079195677:86 , srv217971693:86 , srv297555996:86 , srv310097732:86 , srv319367091:86 , srv400193896:86 , srv428835040:86 , srv45235417:86 , srv617689389:86 , srv636223582:86 , srv726882459:86 , srv78563019:86 , srv1577415130:87 , srv1719034044:87 , srv1849511341:87 , srv1925473384:87 , srv2112592896:87 , srv220673589:87 , srv262367094:87 , srv568231242:87 , srv610376741:87 , srv675694925:87 , srv735854960:87 , srv738340361:87 , srv812226899:87 , srv958489911:87 , srv1339168364:88 , srv1343562138:88 , srv1563154569:88 , srv1793877172:88 , srv1908907265:88 , srv1920593791:88 , srv829686954:88 , srv985159858:88 , srv1222734448:89 , srv1821423719:543 }
2023-09-28 19:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2146364398=51, srv262367094=54, srv874863077=95, srv535155077=75, srv389594742=62, srv217971693=52, srv62040577=80, srv675694925=85, srv1563154569=20, srv415478630=64, srv910626894=97, srv624352859=81, srv908746042=96, srv2075941256=46, srv2127988923=49, srv389043537=61, srv400193896=63, srv1128052962=3, srv735854960=88, srv47963581=71, srv610376741=78, srv568231242=76, srv1339168364=10, srv598064510=77, srv1920593791=40, srv671616411=84, srv1694253782=27, srv78563019=91, srv1440370998=16, srv1045008004=1, srv310097732=57, srv2079195677=47, srv319367091=58, srv1096889474=2, srv38667306=60, srv1215051503=6, srv1719034044=28, srv2026171542=44, srv1382608998=13, srv1585716862=23, srv1610455955=24, srv509515776=74, srv1821423719=33, srv1870476916=38, srv76230643=90, srv1285092984=9, srv1849511341=34, srv1793877172=30, srv428835040=65, srv1577439769=22, srv467077695=68, srv829686954=93, srv1363598059=12, srv47174726=70, srv1862962542=37, srv1384462473=14, srv812226899=92, srv104146462=0, srv464399430=67, srv669376881=83, srv707292966=86, srv1908907265=39, srv1495494503=19, srv1693472033=26, srv726882459=87, srv1343562138=11, srv467547054=69, srv220673589=53, srv292380380=55, srv738340361=89, srv1242937310=8, srv139760886=15, srv1453046460=17, srv2008410997=42, srv1577415130=21, srv36637626=59, srv869951816=94, srv2112592896=48, srv985159858=99, srv1633689953=25, srv1204121728=5, srv489169636=72, srv636223582=82, srv2057159842=45, srv508954217=73, srv1857520447=36, srv2018916427=43, srv2132192012=50, srv1777456876=29, srv45235417=66, srv1185341860=4, srv958489911=98, srv1925473384=41, srv1812551542=32, srv1454991571=18, srv297555996=56, srv1854968203=35, srv617689389=79, srv1800981679=31, srv1222734448=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:53:17,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:53:17,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:53:17,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:53:17,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 19:53:17,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 19:53:17,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 19:53:17,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 19:53:17,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 19:53:17,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 19:53:17,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 19:53:17,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 19:53:17,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 19:53:17,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006217570812329759); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012463189345605802); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006217570812329759); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:17,321 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 19:53:17,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.012372353478457E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006217570812329759); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012463189345605802); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006217570812329759); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 19:53:17,662 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 459 ms to try 5000 different iterations.  Found a solution that moves 257 regions; Going from a computed imbalance of 7.012372353478457E-4 to a new imbalance of 2.0631922002392673E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.028555555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:17,663 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1045008004:83 , srv1339168364:83 , srv1577415130:83 , srv1800981679:83 , srv2008410997:83 , srv45235417:83 , srv675694925:83 , srv707292966:83 , srv958489911:83 , srv104146462:84 , srv1222734448:84 , srv1242937310:84 , srv1693472033:84 , srv1777456876:84 , srv1793877172:84 , srv1812551542:84 , srv1857520447:84 , srv1908907265:84 , srv1920593791:84 , srv1925473384:84 , srv2075941256:84 , srv2079195677:84 , srv2112592896:84 , srv2132192012:84 , srv217971693:84 , srv310097732:84 , srv319367091:84 , srv38667306:84 , srv389043537:84 , srv400193896:84 , srv428835040:84 , srv464399430:84 , srv47174726:84 , srv509515776:84 , srv535155077:84 , srv568231242:84 , srv617689389:84 , srv62040577:84 , srv636223582:84 , srv671616411:84 , srv735854960:84 , srv812226899:84 , srv829686954:84 , srv985159858:84 , srv1096889474:85 , srv1128052962:85 , srv1185341860:85 , srv1215051503:85 , srv1285092984:85 , srv1343562138:85 , srv1363598059:85 , srv1382608998:85 , srv1384462473:85 , srv139760886:85 , srv1440370998:85 , srv1453046460:85 , srv1495494503:85 , srv1563154569:85 , srv1585716862:85 , srv1610455955:85 , srv1633689953:85 , srv1694253782:85 , srv1719034044:85 , srv1849511341:85 , srv1854968203:85 , srv1862962542:85 , srv2018916427:85 , srv2026171542:85 , srv2057159842:85 , srv2127988923:85 , srv2146364398:85 , srv220673589:85 , srv262367094:85 , srv36637626:85 , srv467077695:85 , srv467547054:85 , srv47963581:85 , srv489169636:85 , srv598064510:85 , srv610376741:85 , srv624352859:85 , srv669376881:85 , srv726882459:85 , srv738340361:85 , srv76230643:85 , srv78563019:85 , srv869951816:85 , srv874863077:85 , srv908746042:85 , srv910626894:85 , srv1204121728:86 , srv1454991571:86 , srv1577439769:86 , srv1870476916:86 , srv292380380:86 , srv297555996:86 , srv389594742:86 , srv415478630:86 , srv508954217:86 , srv1821423719:629 }
2023-09-28 19:53:17,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:53:17,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2146364398=51, srv262367094=54, srv874863077=95, srv535155077=75, srv389594742=62, srv217971693=52, srv62040577=80, srv675694925=85, srv1563154569=20, srv415478630=64, srv910626894=97, srv624352859=81, srv908746042=96, srv2075941256=46, srv2127988923=49, srv389043537=61, srv400193896=63, srv1128052962=3, srv735854960=88, srv47963581=71, srv610376741=78, srv568231242=76, srv1339168364=10, srv598064510=77, srv1920593791=40, srv671616411=84, srv1694253782=27, srv78563019=91, srv1440370998=16, srv1045008004=1, srv310097732=57, srv2079195677=47, srv319367091=58, srv1096889474=2, srv38667306=60, srv1215051503=6, srv1719034044=28, srv2026171542=44, srv1382608998=13, srv1585716862=23, srv1610455955=24, srv509515776=74, srv1821423719=33, srv1870476916=38, srv76230643=90, srv1285092984=9, srv1849511341=34, srv1793877172=30, srv428835040=65, srv1577439769=22, srv467077695=68, srv829686954=93, srv1363598059=12, srv47174726=70, srv1862962542=37, srv1384462473=14, srv812226899=92, srv104146462=0, srv464399430=67, srv669376881=83, srv707292966=86, srv1908907265=39, srv1495494503=19, srv1693472033=26, srv726882459=87, srv1343562138=11, srv467547054=69, srv220673589=53, srv292380380=55, srv738340361=89, srv1242937310=8, srv139760886=15, srv1453046460=17, srv2008410997=42, srv1577415130=21, srv36637626=59, srv869951816=94, srv2112592896=48, srv985159858=99, srv1633689953=25, srv1204121728=5, srv489169636=72, srv636223582=82, srv2057159842=45, srv508954217=73, srv1857520447=36, srv2018916427=43, srv2132192012=50, srv1777456876=29, srv45235417=66, srv1185341860=4, srv958489911=98, srv1925473384=41, srv1812551542=32, srv1454991571=18, srv297555996=56, srv1854968203=35, srv617689389=79, srv1800981679=31, srv1222734448=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:53:17,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:53:17,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:53:17,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 19:53:17,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 19:53:17,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 19:53:17,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 19:53:17,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 19:53:17,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 19:53:17,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 19:53:17,695 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 19:53:17,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 19:53:17,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 19:53:17,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 19:53:17,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 19:53:17,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 19:53:17,800 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8832369539786655E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:17,800 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1045008004:83 , srv1339168364:83 , srv1577415130:83 , srv1800981679:83 , srv2008410997:83 , srv45235417:83 , srv675694925:83 , srv707292966:83 , srv958489911:83 , srv104146462:84 , srv1222734448:84 , srv1242937310:84 , srv1693472033:84 , srv1777456876:84 , srv1793877172:84 , srv1812551542:84 , srv1857520447:84 , srv1908907265:84 , srv1920593791:84 , srv1925473384:84 , srv2075941256:84 , srv2079195677:84 , srv2112592896:84 , srv2132192012:84 , srv217971693:84 , srv310097732:84 , srv319367091:84 , srv38667306:84 , srv389043537:84 , srv400193896:84 , srv428835040:84 , srv464399430:84 , srv47174726:84 , srv509515776:84 , srv535155077:84 , srv568231242:84 , srv617689389:84 , srv62040577:84 , srv636223582:84 , srv671616411:84 , srv735854960:84 , srv812226899:84 , srv829686954:84 , srv985159858:84 , srv1096889474:85 , srv1128052962:85 , srv1185341860:85 , srv1215051503:85 , srv1285092984:85 , srv1343562138:85 , srv1363598059:85 , srv1382608998:85 , srv1384462473:85 , srv139760886:85 , srv1440370998:85 , srv1453046460:85 , srv1495494503:85 , srv1563154569:85 , srv1585716862:85 , srv1610455955:85 , srv1633689953:85 , srv1694253782:85 , srv1719034044:85 , srv1849511341:85 , srv1854968203:85 , srv1862962542:85 , srv2018916427:85 , srv2026171542:85 , srv2057159842:85 , srv2127988923:85 , srv2146364398:85 , srv220673589:85 , srv262367094:85 , srv36637626:85 , srv467077695:85 , srv467547054:85 , srv47963581:85 , srv489169636:85 , srv598064510:85 , srv610376741:85 , srv624352859:85 , srv669376881:85 , srv726882459:85 , srv738340361:85 , srv76230643:85 , srv78563019:85 , srv869951816:85 , srv874863077:85 , srv908746042:85 , srv910626894:85 , srv1204121728:86 , srv1454991571:86 , srv1577439769:86 , srv1870476916:86 , srv292380380:86 , srv297555996:86 , srv389594742:86 , srv415478630:86 , srv508954217:86 , srv1821423719:629 }
2023-09-28 19:53:17,830 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=293 (was 293), ProcessCount=171 (was 171), AvailableMemoryMB=5956 (was 6568)
2023-09-28 19:53:17,838 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=171, AvailableMemoryMB=5956
2023-09-28 19:53:17,839 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-28 19:53:17,839 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-28 19:53:17,851 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1051701218:84 , srv1085132483:84 , srv1122511989:84 , srv1143503867:84 , srv1150471141:84 , srv1153603846:84 , srv1228976529:84 , srv1448111342:84 , srv1491379409:84 , srv1692476615:84 , srv1700719816:84 , srv1781810123:84 , srv2091752078:84 , srv2101783543:84 , srv21031501:84 , srv281938327:84 , srv452659619:84 , srv459299399:84 , srv474595540:84 , srv570972926:84 , srv582723539:84 , srv595389951:84 , srv6879247:84 , srv723496694:84 , srv838079717:84 , srv873577176:84 , srv920722743:84 , srv992500191:84 , srv997731124:84 , srv727498220:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-28 19:53:17,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:53:17,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1781810123=11, srv1143503867=3, srv723496694=23, srv1448111342=7, srv1085132483=1, srv1228976529=6, srv21031501=14, srv1692476615=9, srv474595540=18, srv1491379409=8, srv1700719816=10, srv459299399=17, srv1051701218=0, srv452659619=16, srv2101783543=13, srv873577176=26, srv1122511989=2, srv2091752078=12, srv838079717=25, srv920722743=27, srv997731124=29, srv582723539=20, srv992500191=28, srv281938327=15, srv595389951=21, srv570972926=19, srv6879247=22, srv1153603846=5, srv1150471141=4, srv727498220=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:53:17,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:53:17,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:53:17,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:53:17,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-28 19:53:17,895 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1248590168753752, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:53:17,895 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912586469949431, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1248590168753752, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-28 19:55:17,873 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5696128 different iterations.  Found a solution that moves 1864 regions; Going from a computed imbalance of 0.9912586469949431 to a new imbalance of 4.3506689887128686E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6903703703703704, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:55:17,878 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1051701218:90 , srv1085132483:90 , srv1122511989:90 , srv1143503867:90 , srv1150471141:90 , srv1153603846:90 , srv1228976529:90 , srv1448111342:90 , srv1491379409:90 , srv1692476615:90 , srv1700719816:90 , srv1781810123:90 , srv2091752078:90 , srv2101783543:90 , srv21031501:90 , srv281938327:90 , srv452659619:90 , srv459299399:90 , srv474595540:90 , srv570972926:90 , srv582723539:90 , srv595389951:90 , srv6879247:90 , srv723496694:90 , srv727498220:90 , srv838079717:90 , srv873577176:90 , srv920722743:90 , srv992500191:90 , srv997731124:90 }
2023-09-28 19:55:17,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 19:55:17,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1781810123=11, srv1143503867=3, srv723496694=23, srv1448111342=7, srv1085132483=1, srv1228976529=6, srv21031501=14, srv1692476615=9, srv474595540=18, srv1491379409=8, srv1700719816=10, srv459299399=17, srv1051701218=0, srv452659619=16, srv2101783543=13, srv873577176=26, srv1122511989=2, srv2091752078=12, srv838079717=25, srv920722743=27, srv997731124=29, srv582723539=20, srv992500191=28, srv281938327=15, srv595389951=21, srv570972926=19, srv6879247=22, srv1153603846=5, srv1150471141=4, srv727498220=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 19:55:17,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 19:55:17,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 19:55:17,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 19:55:17,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 19:55:17,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 19:55:17,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 19:55:17,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 19:55:17,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 19:55:17,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 19:55:17,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-28 19:55:17,897 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 19:55:17,897 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1051701218:90 , srv1085132483:90 , srv1122511989:90 , srv1143503867:90 , srv1150471141:90 , srv1153603846:90 , srv1228976529:90 , srv1448111342:90 , srv1491379409:90 , srv1692476615:90 , srv1700719816:90 , srv1781810123:90 , srv2091752078:90 , srv2101783543:90 , srv21031501:90 , srv281938327:90 , srv452659619:90 , srv459299399:90 , srv474595540:90 , srv570972926:90 , srv582723539:90 , srv595389951:90 , srv6879247:90 , srv723496694:90 , srv727498220:90 , srv838079717:90 , srv873577176:90 , srv920722743:90 , srv992500191:90 , srv997731124:90 }
2023-09-28 19:55:17,909 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 293), ProcessCount=171 (was 171), AvailableMemoryMB=4722 (was 5956)