2023-08-29 06:53:12,898 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-29 06:53:13,221 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-29 06:53:13,347 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-29 06:53:13,425 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-29 06:53:13,436 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-29 06:53:13,452 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=173, AvailableMemoryMB=11738
2023-08-29 06:53:13,455 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-29 06:53:13,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-29 06:53:13,612 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1013004433:84 , srv1053855341:84 , srv1111673176:84 , srv1137153591:84 , srv1174887935:84 , srv1184191996:84 , srv118797581:84 , srv1219081066:84 , srv1224376273:84 , srv1233959046:84 , srv1249670557:84 , srv1256887191:84 , srv125886204:84 , srv1268409062:84 , srv1314716747:84 , srv1331024976:84 , srv1365076947:84 , srv1385175581:84 , srv1396137749:84 , srv1401178041:84 , srv1405921438:84 , srv1409527033:84 , srv1424832990:84 , srv1442519227:84 , srv152231517:84 , srv1530618137:84 , srv1533871889:84 , srv1555961745:84 , srv1572657520:84 , srv1594659904:84 , srv1613285723:84 , srv1617268170:84 , srv169031643:84 , srv1696599739:84 , srv1697622714:84 , srv1720447023:84 , srv1728528846:84 , srv1734971265:84 , srv1790468161:84 , srv1830619666:84 , srv1840042380:84 , srv1844199457:84 , srv18680676:84 , srv1903557526:84 , srv1927855014:84 , srv1930455589:84 , srv1944126494:84 , srv2011842805:84 , srv2012170152:84 , srv2012844009:84 , srv2022907738:84 , srv2025457117:84 , srv202680680:84 , srv2041479340:84 , srv2042797001:84 , srv2052770960:84 , srv2073188445:84 , srv2130244415:84 , srv2146030983:84 , srv227813648:84 , srv256558977:84 , srv303308964:84 , srv304902250:84 , srv330915632:84 , srv337087481:84 , srv36111176:84 , srv40353672:84 , srv40816866:84 , srv413356596:84 , srv422574200:84 , srv433229774:84 , srv442634320:84 , srv455309597:84 , srv472256284:84 , srv485231909:84 , srv493836863:84 , srv502394778:84 , srv553185344:84 , srv556440391:84 , srv558173516:84 , srv582313713:84 , srv616951127:84 , srv626135369:84 , srv646403040:84 , srv647034292:84 , srv658263686:84 , srv664034567:84 , srv696127121:84 , srv714749416:84 , srv753053655:84 , srv763669414:84 , srv783896334:84 , srv794208803:84 , srv812809203:84 , srv839821426:84 , srv873055600:84 , srv896506448:84 , srv918550583:84 , srv986748405:84 , srv1353639333:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-29 06:53:13,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:53:13,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2012170152=49, srv714749416=89, srv896506448=97, srv1728528846=37, srv1268409062=13, srv1617268170=32, srv2025457117=52, srv647034292=85, srv1256887191=11, srv2022907738=51, srv422574200=70, srv794208803=93, srv118797581=6, srv1696599739=34, srv783896334=92, srv125886204=12, srv472256284=74, srv658263686=86, srv1353639333=16, srv1594659904=30, srv1013004433=0, srv873055600=96, srv2146030983=59, srv556440391=79, srv1174887935=4, srv1405921438=21, srv413356596=69, srv839821426=95, srv1442519227=24, srv1903557526=44, srv2011842805=48, srv152231517=25, srv1401178041=20, srv1790468161=39, srv2041479340=54, srv553185344=78, srv1396137749=19, srv442634320=72, srv2012844009=50, srv433229774=71, srv1233959046=9, srv1944126494=47, srv812809203=94, srv646403040=84, srv202680680=53, srv1555961745=28, srv1720447023=36, srv1111673176=2, srv1314716747=14, srv304902250=63, srv763669414=91, srv337087481=65, srv1219081066=7, srv626135369=83, srv1734971265=38, srv1331024976=15, srv753053655=90, srv1365076947=17, srv1533871889=27, srv455309597=73, srv986748405=99, srv1224376273=8, srv1409527033=22, srv1137153591=3, srv330915632=64, srv1184191996=5, srv1697622714=35, srv2052770960=56, srv169031643=33, srv696127121=88, srv1830619666=40, srv2042797001=55, srv502394778=77, srv256558977=61, srv40353672=67, srv1424832990=23, srv1530618137=26, srv485231909=75, srv1844199457=42, srv18680676=43, srv303308964=62, srv582313713=81, srv918550583=98, srv1572657520=29, srv558173516=80, srv40816866=68, srv1249670557=10, srv1053855341=1, srv2130244415=58, srv616951127=82, srv664034567=87, srv227813648=60, srv1385175581=18, srv1930455589=46, srv493836863=76, srv1927855014=45, srv2073188445=57, srv1613285723=31, srv36111176=66, srv1840042380=41} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:53:13,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:53:13,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:53:13,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:53:13,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-29 06:53:13,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-29 06:53:13,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-29 06:53:13,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-29 06:53:13,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-29 06:53:13,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-29 06:53:13,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-29 06:53:13,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-29 06:53:13,958 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:13,958 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-29 06:53:13,959 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-29 06:53:14,320 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 577 ms to try 5000 different iterations.  Found a solution that moves 4688 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18182138742556345. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040300343924264674, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05860135636188211, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5208888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040300343924264674, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24666666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:14,333 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2022907738:82 , srv202680680:82 , srv626135369:82 , srv763669414:82 , srv1268409062:83 , srv1385175581:83 , srv1424832990:83 , srv1617268170:83 , srv18680676:83 , srv2146030983:83 , srv256558977:83 , srv647034292:83 , srv839821426:83 , srv1053855341:84 , srv1111673176:84 , srv1365076947:84 , srv1533871889:84 , srv1555961745:84 , srv1572657520:84 , srv1594659904:84 , srv1728528846:84 , srv2041479340:84 , srv40353672:84 , srv442634320:84 , srv472256284:84 , srv493836863:84 , srv616951127:84 , srv118797581:85 , srv1219081066:85 , srv125886204:85 , srv1401178041:85 , srv1409527033:85 , srv152231517:85 , srv1697622714:85 , srv1720447023:85 , srv1790468161:85 , srv1840042380:85 , srv1844199457:85 , srv1927855014:85 , srv1930455589:85 , srv2042797001:85 , srv2073188445:85 , srv502394778:85 , srv658263686:85 , srv1174887935:86 , srv1696599739:86 , srv2012170152:86 , srv2025457117:86 , srv303308964:86 , srv433229774:86 , srv582313713:86 , srv696127121:86 , srv753053655:86 , srv812809203:86 , srv1137153591:87 , srv1184191996:87 , srv1233959046:87 , srv1249670557:87 , srv1256887191:87 , srv1314716747:87 , srv1530618137:87 , srv1903557526:87 , srv2130244415:87 , srv227813648:87 , srv337087481:87 , srv413356596:87 , srv422574200:87 , srv455309597:87 , srv918550583:87 , srv986748405:87 , srv1331024976:88 , srv1396137749:88 , srv1734971265:88 , srv1830619666:88 , srv1944126494:88 , srv304902250:88 , srv330915632:88 , srv36111176:88 , srv556440391:88 , srv873055600:88 , srv896506448:88 , srv1224376273:89 , srv1405921438:89 , srv1613285723:89 , srv664034567:89 , srv794208803:89 , srv1013004433:90 , srv1442519227:90 , srv2012844009:90 , srv2052770960:90 , srv485231909:90 , srv553185344:90 , srv646403040:90 , srv2011842805:91 , srv169031643:92 , srv558173516:92 , srv714749416:95 , srv783896334:95 , srv40816866:96 , srv1353639333:448 }
2023-08-29 06:53:14,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:53:14,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2012170152=49, srv714749416=89, srv896506448=97, srv1728528846=37, srv1268409062=13, srv1617268170=32, srv2025457117=52, srv647034292=85, srv1256887191=11, srv2022907738=51, srv422574200=70, srv794208803=93, srv118797581=6, srv1696599739=34, srv783896334=92, srv125886204=12, srv472256284=74, srv658263686=86, srv1353639333=16, srv1594659904=30, srv1013004433=0, srv873055600=96, srv2146030983=59, srv556440391=79, srv1174887935=4, srv1405921438=21, srv413356596=69, srv839821426=95, srv1442519227=24, srv1903557526=44, srv2011842805=48, srv152231517=25, srv1401178041=20, srv1790468161=39, srv2041479340=54, srv553185344=78, srv1396137749=19, srv442634320=72, srv2012844009=50, srv433229774=71, srv1233959046=9, srv1944126494=47, srv812809203=94, srv646403040=84, srv202680680=53, srv1555961745=28, srv1720447023=36, srv1111673176=2, srv1314716747=14, srv304902250=63, srv763669414=91, srv337087481=65, srv1219081066=7, srv626135369=83, srv1734971265=38, srv1331024976=15, srv753053655=90, srv1365076947=17, srv1533871889=27, srv455309597=73, srv986748405=99, srv1224376273=8, srv1409527033=22, srv1137153591=3, srv330915632=64, srv1184191996=5, srv1697622714=35, srv2052770960=56, srv169031643=33, srv696127121=88, srv1830619666=40, srv2042797001=55, srv502394778=77, srv256558977=61, srv40353672=67, srv1424832990=23, srv1530618137=26, srv485231909=75, srv1844199457=42, srv18680676=43, srv303308964=62, srv582313713=81, srv918550583=98, srv1572657520=29, srv558173516=80, srv40816866=68, srv1249670557=10, srv1053855341=1, srv2130244415=58, srv616951127=82, srv664034567=87, srv227813648=60, srv1385175581=18, srv1930455589=46, srv493836863=76, srv1927855014=45, srv2073188445=57, srv1613285723=31, srv36111176=66, srv1840042380=41} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:53:14,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:53:14,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:53:14,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:53:14,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:53:14,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:53:14,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-29 06:53:14,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-29 06:53:14,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-29 06:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-29 06:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-29 06:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-29 06:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-29 06:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-29 06:53:14,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-29 06:53:14,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-29 06:53:14,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-29 06:53:14,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-29 06:53:14,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-29 06:53:14,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-29 06:53:14,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-29 06:53:14,546 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040300343924264674, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05860135636188211, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040300343924264674, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24666666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:14,546 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-29 06:53:14,547 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18178856134795762, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040300343924264674, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05860135636188211, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040300343924264674, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24666666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-29 06:53:14,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 555 ms to try 5000 different iterations.  Found a solution that moves 2671 regions; Going from a computed imbalance of 0.18178856134795762 to a new imbalance of 0.01586597136839462. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022511584082034942); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034780052851005605, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29677777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022511584082034942); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:14,963 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv40816866:75 , srv714749416:77 , srv169031643:78 , srv783896334:78 , srv2052770960:80 , srv485231909:80 , srv558173516:80 , srv1013004433:81 , srv1224376273:81 , srv1734971265:81 , srv2011842805:81 , srv1137153591:82 , srv646403040:82 , srv896506448:82 , srv1256887191:83 , srv1331024976:83 , srv1405921438:83 , srv1442519227:83 , srv2012844009:83 , srv227813648:83 , srv330915632:83 , srv337087481:83 , srv556440391:83 , srv1396137749:84 , srv1613285723:84 , srv1830619666:84 , srv2130244415:84 , srv455309597:84 , srv873055600:84 , srv1174887935:85 , srv1184191996:85 , srv1249670557:85 , srv1530618137:85 , srv1790468161:85 , srv1944126494:85 , srv304902250:85 , srv664034567:85 , srv753053655:85 , srv1219081066:86 , srv1233959046:86 , srv152231517:86 , srv1533871889:86 , srv1696599739:86 , srv1720447023:86 , srv1840042380:86 , srv18680676:86 , srv1903557526:86 , srv2042797001:86 , srv36111176:86 , srv422574200:86 , srv433229774:86 , srv442634320:86 , srv553185344:86 , srv696127121:86 , srv794208803:86 , srv812809203:86 , srv1053855341:87 , srv1111673176:87 , srv118797581:87 , srv125886204:87 , srv1314716747:87 , srv1401178041:87 , srv1424832990:87 , srv1697622714:87 , srv1728528846:87 , srv1927855014:87 , srv2012170152:87 , srv2025457117:87 , srv2073188445:87 , srv40353672:87 , srv413356596:87 , srv502394778:87 , srv616951127:87 , srv763669414:87 , srv918550583:87 , srv1572657520:88 , srv1594659904:88 , srv1844199457:88 , srv2146030983:88 , srv303308964:88 , srv472256284:88 , srv493836863:88 , srv658263686:88 , srv1365076947:89 , srv1385175581:89 , srv1409527033:89 , srv1555961745:89 , srv1617268170:89 , srv1930455589:89 , srv2022907738:89 , srv202680680:89 , srv582313713:89 , srv1268409062:90 , srv256558977:90 , srv647034292:90 , srv986748405:90 , srv2041479340:91 , srv626135369:91 , srv839821426:92 , srv1353639333:526 }
2023-08-29 06:53:14,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:53:14,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2012170152=49, srv714749416=89, srv896506448=97, srv1728528846=37, srv1268409062=13, srv1617268170=32, srv2025457117=52, srv647034292=85, srv1256887191=11, srv2022907738=51, srv422574200=70, srv794208803=93, srv118797581=6, srv1696599739=34, srv783896334=92, srv125886204=12, srv472256284=74, srv658263686=86, srv1353639333=16, srv1594659904=30, srv1013004433=0, srv873055600=96, srv2146030983=59, srv556440391=79, srv1174887935=4, srv1405921438=21, srv413356596=69, srv839821426=95, srv1442519227=24, srv1903557526=44, srv2011842805=48, srv152231517=25, srv1401178041=20, srv1790468161=39, srv2041479340=54, srv553185344=78, srv1396137749=19, srv442634320=72, srv2012844009=50, srv433229774=71, srv1233959046=9, srv1944126494=47, srv812809203=94, srv646403040=84, srv202680680=53, srv1555961745=28, srv1720447023=36, srv1111673176=2, srv1314716747=14, srv304902250=63, srv763669414=91, srv337087481=65, srv1219081066=7, srv626135369=83, srv1734971265=38, srv1331024976=15, srv753053655=90, srv1365076947=17, srv1533871889=27, srv455309597=73, srv986748405=99, srv1224376273=8, srv1409527033=22, srv1137153591=3, srv330915632=64, srv1184191996=5, srv1697622714=35, srv2052770960=56, srv169031643=33, srv696127121=88, srv1830619666=40, srv2042797001=55, srv502394778=77, srv256558977=61, srv40353672=67, srv1424832990=23, srv1530618137=26, srv485231909=75, srv1844199457=42, srv18680676=43, srv303308964=62, srv582313713=81, srv918550583=98, srv1572657520=29, srv558173516=80, srv40816866=68, srv1249670557=10, srv1053855341=1, srv2130244415=58, srv616951127=82, srv664034567=87, srv227813648=60, srv1385175581=18, srv1930455589=46, srv493836863=76, srv1927855014=45, srv2073188445=57, srv1613285723=31, srv36111176=66, srv1840042380=41} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:53:15,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:53:15,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:53:15,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:53:15,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-29 06:53:15,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-29 06:53:15,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-29 06:53:15,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-29 06:53:15,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-29 06:53:15,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-29 06:53:15,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-29 06:53:15,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-29 06:53:15,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-29 06:53:15,153 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022511584082034942); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034780052851005605, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022511584082034942); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:15,153 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-29 06:53:15,153 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015847268626652904, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022511584082034942); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.034780052851005605, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022511584082034942); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-29 06:53:15,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 390 ms to try 5000 different iterations.  Found a solution that moves 1045 regions; Going from a computed imbalance of 0.015847268626652904 to a new imbalance of 0.0012622320514495257. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0068910969995776596); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00974420314594759); MoveCostFunction : (multiplier=7.0, imbalance=0.11611111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0068910969995776596); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:15,424 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv986748405:81 , srv1314716747:82 , srv553185344:82 , srv582313713:82 , srv1409527033:83 , srv1930455589:83 , srv2041479340:83 , srv303308964:83 , srv36111176:83 , srv626135369:83 , srv839821426:83 , srv918550583:83 , srv1233959046:84 , srv1401178041:84 , srv1442519227:84 , srv1555961745:84 , srv1613285723:84 , srv1903557526:84 , srv2012844009:84 , srv2025457117:84 , srv422574200:84 , srv658263686:84 , srv794208803:84 , srv812809203:84 , srv1174887935:85 , srv118797581:85 , srv1249670557:85 , srv125886204:85 , srv1268409062:85 , srv1365076947:85 , srv1385175581:85 , srv1530618137:85 , srv1594659904:85 , srv1617268170:85 , srv1696599739:85 , srv1844199457:85 , srv1927855014:85 , srv1944126494:85 , srv2011842805:85 , srv2012170152:85 , srv202680680:85 , srv2073188445:85 , srv256558977:85 , srv304902250:85 , srv330915632:85 , srv40353672:85 , srv413356596:85 , srv433229774:85 , srv472256284:85 , srv493836863:85 , srv502394778:85 , srv646403040:85 , srv647034292:85 , srv664034567:85 , srv696127121:85 , srv783896334:85 , srv1053855341:86 , srv1111673176:86 , srv1184191996:86 , srv1396137749:86 , srv1405921438:86 , srv1572657520:86 , srv1697622714:86 , srv1720447023:86 , srv1728528846:86 , srv1830619666:86 , srv1840042380:86 , srv2130244415:86 , srv2146030983:86 , srv40816866:86 , srv558173516:86 , srv616951127:86 , srv714749416:86 , srv873055600:86 , srv1013004433:87 , srv1219081066:87 , srv1224376273:87 , srv1256887191:87 , srv1331024976:87 , srv152231517:87 , srv2022907738:87 , srv2042797001:87 , srv2052770960:87 , srv337087481:87 , srv442634320:87 , srv455309597:87 , srv485231909:87 , srv556440391:87 , srv753053655:87 , srv1137153591:88 , srv1424832990:88 , srv1533871889:88 , srv1734971265:88 , srv227813648:88 , srv763669414:88 , srv896506448:88 , srv169031643:89 , srv1790468161:89 , srv18680676:89 , srv1353639333:545 }
2023-08-29 06:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:53:15,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2012170152=49, srv714749416=89, srv896506448=97, srv1728528846=37, srv1268409062=13, srv1617268170=32, srv2025457117=52, srv647034292=85, srv1256887191=11, srv2022907738=51, srv422574200=70, srv794208803=93, srv118797581=6, srv1696599739=34, srv783896334=92, srv125886204=12, srv472256284=74, srv658263686=86, srv1353639333=16, srv1594659904=30, srv1013004433=0, srv873055600=96, srv2146030983=59, srv556440391=79, srv1174887935=4, srv1405921438=21, srv413356596=69, srv839821426=95, srv1442519227=24, srv1903557526=44, srv2011842805=48, srv152231517=25, srv1401178041=20, srv1790468161=39, srv2041479340=54, srv553185344=78, srv1396137749=19, srv442634320=72, srv2012844009=50, srv433229774=71, srv1233959046=9, srv1944126494=47, srv812809203=94, srv646403040=84, srv202680680=53, srv1555961745=28, srv1720447023=36, srv1111673176=2, srv1314716747=14, srv304902250=63, srv763669414=91, srv337087481=65, srv1219081066=7, srv626135369=83, srv1734971265=38, srv1331024976=15, srv753053655=90, srv1365076947=17, srv1533871889=27, srv455309597=73, srv986748405=99, srv1224376273=8, srv1409527033=22, srv1137153591=3, srv330915632=64, srv1184191996=5, srv1697622714=35, srv2052770960=56, srv169031643=33, srv696127121=88, srv1830619666=40, srv2042797001=55, srv502394778=77, srv256558977=61, srv40353672=67, srv1424832990=23, srv1530618137=26, srv485231909=75, srv1844199457=42, srv18680676=43, srv303308964=62, srv582313713=81, srv918550583=98, srv1572657520=29, srv558173516=80, srv40816866=68, srv1249670557=10, srv1053855341=1, srv2130244415=58, srv616951127=82, srv664034567=87, srv227813648=60, srv1385175581=18, srv1930455589=46, srv493836863=76, srv1927855014=45, srv2073188445=57, srv1613285723=31, srv36111176=66, srv1840042380=41} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:53:15,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:53:15,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:53:15,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-29 06:53:15,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-29 06:53:15,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-29 06:53:15,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-29 06:53:15,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-29 06:53:15,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-29 06:53:15,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-29 06:53:15,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-29 06:53:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-29 06:53:15,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-29 06:53:15,574 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0068910969995776596); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00974420314594759); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0068910969995776596); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:15,574 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-29 06:53:15,575 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0012549148050548826, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0068910969995776596); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00974420314594759); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0068910969995776596); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-29 06:53:15,969 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 504 ms to try 5000 different iterations.  Found a solution that moves 248 regions; Going from a computed imbalance of 0.0012549148050548826 to a new imbalance of 2.1058338467370438E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.027555555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:15,970 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv413356596:83 , srv558173516:83 , srv664034567:83 , srv794208803:83 , srv1013004433:84 , srv1184191996:84 , srv125886204:84 , srv1365076947:84 , srv1396137749:84 , srv1405921438:84 , srv1409527033:84 , srv1424832990:84 , srv152231517:84 , srv1533871889:84 , srv1572657520:84 , srv169031643:84 , srv1697622714:84 , srv1790468161:84 , srv1830619666:84 , srv18680676:84 , srv1903557526:84 , srv1944126494:84 , srv2012170152:84 , srv2022907738:84 , srv2041479340:84 , srv2042797001:84 , srv2052770960:84 , srv227813648:84 , srv256558977:84 , srv304902250:84 , srv553185344:84 , srv556440391:84 , srv646403040:84 , srv647034292:84 , srv696127121:84 , srv714749416:84 , srv753053655:84 , srv783896334:84 , srv873055600:84 , srv896506448:84 , srv986748405:84 , srv1053855341:85 , srv1111673176:85 , srv1137153591:85 , srv118797581:85 , srv1219081066:85 , srv1224376273:85 , srv1233959046:85 , srv1249670557:85 , srv1256887191:85 , srv1268409062:85 , srv1331024976:85 , srv1385175581:85 , srv1442519227:85 , srv1530618137:85 , srv1555961745:85 , srv1594659904:85 , srv1613285723:85 , srv1617268170:85 , srv1696599739:85 , srv1720447023:85 , srv1728528846:85 , srv1734971265:85 , srv1840042380:85 , srv1844199457:85 , srv1927855014:85 , srv1930455589:85 , srv2011842805:85 , srv2012844009:85 , srv2025457117:85 , srv202680680:85 , srv2073188445:85 , srv2130244415:85 , srv2146030983:85 , srv303308964:85 , srv337087481:85 , srv36111176:85 , srv40816866:85 , srv422574200:85 , srv433229774:85 , srv442634320:85 , srv455309597:85 , srv472256284:85 , srv493836863:85 , srv502394778:85 , srv582313713:85 , srv616951127:85 , srv658263686:85 , srv763669414:85 , srv839821426:85 , srv918550583:85 , srv1174887935:86 , srv1314716747:86 , srv1401178041:86 , srv330915632:86 , srv40353672:86 , srv485231909:86 , srv626135369:86 , srv812809203:86 , srv1353639333:622 }
2023-08-29 06:53:15,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:53:15,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2012170152=49, srv714749416=89, srv896506448=97, srv1728528846=37, srv1268409062=13, srv1617268170=32, srv2025457117=52, srv647034292=85, srv1256887191=11, srv2022907738=51, srv422574200=70, srv794208803=93, srv118797581=6, srv1696599739=34, srv783896334=92, srv125886204=12, srv472256284=74, srv658263686=86, srv1353639333=16, srv1594659904=30, srv1013004433=0, srv873055600=96, srv2146030983=59, srv556440391=79, srv1174887935=4, srv1405921438=21, srv413356596=69, srv839821426=95, srv1442519227=24, srv1903557526=44, srv2011842805=48, srv152231517=25, srv1401178041=20, srv1790468161=39, srv2041479340=54, srv553185344=78, srv1396137749=19, srv442634320=72, srv2012844009=50, srv433229774=71, srv1233959046=9, srv1944126494=47, srv812809203=94, srv646403040=84, srv202680680=53, srv1555961745=28, srv1720447023=36, srv1111673176=2, srv1314716747=14, srv304902250=63, srv763669414=91, srv337087481=65, srv1219081066=7, srv626135369=83, srv1734971265=38, srv1331024976=15, srv753053655=90, srv1365076947=17, srv1533871889=27, srv455309597=73, srv986748405=99, srv1224376273=8, srv1409527033=22, srv1137153591=3, srv330915632=64, srv1184191996=5, srv1697622714=35, srv2052770960=56, srv169031643=33, srv696127121=88, srv1830619666=40, srv2042797001=55, srv502394778=77, srv256558977=61, srv40353672=67, srv1424832990=23, srv1530618137=26, srv485231909=75, srv1844199457=42, srv18680676=43, srv303308964=62, srv582313713=81, srv918550583=98, srv1572657520=29, srv558173516=80, srv40816866=68, srv1249670557=10, srv1053855341=1, srv2130244415=58, srv616951127=82, srv664034567=87, srv227813648=60, srv1385175581=18, srv1930455589=46, srv493836863=76, srv1927855014=45, srv2073188445=57, srv1613285723=31, srv36111176=66, srv1840042380=41} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:53:15,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:53:15,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:53:15,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:53:15,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-29 06:53:15,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-29 06:53:15,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-29 06:53:15,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-29 06:53:15,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-29 06:53:15,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-29 06:53:15,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-29 06:53:16,102 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.932180535170393E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:16,102 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv413356596:83 , srv558173516:83 , srv664034567:83 , srv794208803:83 , srv1013004433:84 , srv1184191996:84 , srv125886204:84 , srv1365076947:84 , srv1396137749:84 , srv1405921438:84 , srv1409527033:84 , srv1424832990:84 , srv152231517:84 , srv1533871889:84 , srv1572657520:84 , srv169031643:84 , srv1697622714:84 , srv1790468161:84 , srv1830619666:84 , srv18680676:84 , srv1903557526:84 , srv1944126494:84 , srv2012170152:84 , srv2022907738:84 , srv2041479340:84 , srv2042797001:84 , srv2052770960:84 , srv227813648:84 , srv256558977:84 , srv304902250:84 , srv553185344:84 , srv556440391:84 , srv646403040:84 , srv647034292:84 , srv696127121:84 , srv714749416:84 , srv753053655:84 , srv783896334:84 , srv873055600:84 , srv896506448:84 , srv986748405:84 , srv1053855341:85 , srv1111673176:85 , srv1137153591:85 , srv118797581:85 , srv1219081066:85 , srv1224376273:85 , srv1233959046:85 , srv1249670557:85 , srv1256887191:85 , srv1268409062:85 , srv1331024976:85 , srv1385175581:85 , srv1442519227:85 , srv1530618137:85 , srv1555961745:85 , srv1594659904:85 , srv1613285723:85 , srv1617268170:85 , srv1696599739:85 , srv1720447023:85 , srv1728528846:85 , srv1734971265:85 , srv1840042380:85 , srv1844199457:85 , srv1927855014:85 , srv1930455589:85 , srv2011842805:85 , srv2012844009:85 , srv2025457117:85 , srv202680680:85 , srv2073188445:85 , srv2130244415:85 , srv2146030983:85 , srv303308964:85 , srv337087481:85 , srv36111176:85 , srv40816866:85 , srv422574200:85 , srv433229774:85 , srv442634320:85 , srv455309597:85 , srv472256284:85 , srv493836863:85 , srv502394778:85 , srv582313713:85 , srv616951127:85 , srv658263686:85 , srv763669414:85 , srv839821426:85 , srv918550583:85 , srv1174887935:86 , srv1314716747:86 , srv1401178041:86 , srv330915632:86 , srv40353672:86 , srv485231909:86 , srv626135369:86 , srv812809203:86 , srv1353639333:622 }
2023-08-29 06:53:16,136 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=374 (was 337) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=10949 (was 11738)
2023-08-29 06:53:16,147 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=374, ProcessCount=173, AvailableMemoryMB=10948
2023-08-29 06:53:16,148 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-29 06:53:16,148 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-29 06:53:16,161 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000934096:84 , srv1027560057:84 , srv1154471180:84 , srv1202421543:84 , srv1290586312:84 , srv1301968780:84 , srv1308821497:84 , srv1366824383:84 , srv1413077382:84 , srv1584091926:84 , srv1695520693:84 , srv1739845609:84 , srv1790040133:84 , srv1804151828:84 , srv1838225633:84 , srv188167613:84 , srv1904822330:84 , srv1908564433:84 , srv1910837849:84 , srv201137555:84 , srv356885745:84 , srv437174209:84 , srv556607794:84 , srv566106047:84 , srv610686602:84 , srv779994127:84 , srv781671589:84 , srv92415100:84 , srv953770464:84 , srv2021362192:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-29 06:53:16,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:53:16,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1695520693=10, srv610686602=25, srv1366824383=7, srv1584091926=9, srv556607794=23, srv1838225633=14, srv1290586312=4, srv1413077382=8, srv201137555=19, srv953770464=29, srv1308821497=6, srv1154471180=2, srv1790040133=12, srv437174209=22, srv1000934096=0, srv1301968780=5, srv2021362192=20, srv1739845609=11, srv188167613=15, srv1027560057=1, srv1202421543=3, srv1904822330=16, srv1908564433=17, srv356885745=21, srv566106047=24, srv779994127=26, srv92415100=28, srv1910837849=18, srv781671589=27, srv1804151828=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:53:16,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:53:16,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:53:16,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:53:16,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:53:16,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-29 06:53:16,202 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1031340334631055, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:53:16,202 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912518015236087, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1031340334631055, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-29 06:55:16,183 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6276419 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912518015236087 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:55:16,192 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1000934096:90 , srv1027560057:90 , srv1154471180:90 , srv1202421543:90 , srv1290586312:90 , srv1301968780:90 , srv1308821497:90 , srv1366824383:90 , srv1413077382:90 , srv1584091926:90 , srv1695520693:90 , srv1739845609:90 , srv1790040133:90 , srv1804151828:90 , srv1838225633:90 , srv188167613:90 , srv1904822330:90 , srv1908564433:90 , srv1910837849:90 , srv201137555:90 , srv2021362192:90 , srv356885745:90 , srv437174209:90 , srv556607794:90 , srv566106047:90 , srv610686602:90 , srv779994127:90 , srv781671589:90 , srv92415100:90 , srv953770464:90 }
2023-08-29 06:55:16,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-29 06:55:16,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1695520693=10, srv610686602=25, srv1366824383=7, srv1584091926=9, srv556607794=23, srv1838225633=14, srv1290586312=4, srv1413077382=8, srv201137555=19, srv953770464=29, srv1308821497=6, srv1154471180=2, srv1790040133=12, srv437174209=22, srv1000934096=0, srv1301968780=5, srv2021362192=20, srv1739845609=11, srv188167613=15, srv1027560057=1, srv1202421543=3, srv1904822330=16, srv1908564433=17, srv356885745=21, srv566106047=24, srv779994127=26, srv92415100=28, srv1910837849=18, srv781671589=27, srv1804151828=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-29 06:55:16,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-29 06:55:16,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-29 06:55:16,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-29 06:55:16,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-29 06:55:16,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-29 06:55:16,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-29 06:55:16,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-29 06:55:16,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-29 06:55:16,215 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1000934096:90 , srv1027560057:90 , srv1154471180:90 , srv1202421543:90 , srv1290586312:90 , srv1301968780:90 , srv1308821497:90 , srv1366824383:90 , srv1413077382:90 , srv1584091926:90 , srv1695520693:90 , srv1739845609:90 , srv1790040133:90 , srv1804151828:90 , srv1838225633:90 , srv188167613:90 , srv1904822330:90 , srv1908564433:90 , srv1910837849:90 , srv201137555:90 , srv2021362192:90 , srv356885745:90 , srv437174209:90 , srv556607794:90 , srv566106047:90 , srv610686602:90 , srv779994127:90 , srv781671589:90 , srv92415100:90 , srv953770464:90 }
2023-08-29 06:55:16,230 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=191 (was 374), ProcessCount=171 (was 173), AvailableMemoryMB=10862 (was 10948)