2024-04-22 01:53:51,981 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-04-22 01:53:52,298 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-04-22 01:53:52,402 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-04-22 01:53:52,472 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-22 01:53:52,480 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-22 01:53:52,495 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=248, ProcessCount=168, AvailableMemoryMB=12544
2024-04-22 01:53:52,498 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-22 01:53:52,498 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-04-22 01:53:52,634 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1003487539:84 , srv1004563329:84 , srv1092101582:84 , srv109487103:84 , srv111111552:84 , srv1121319703:84 , srv1134306238:84 , srv1166573466:84 , srv1178909593:84 , srv11915219:84 , srv1200549333:84 , srv1211008246:84 , srv1213489394:84 , srv125742280:84 , srv1274863374:84 , srv1311545670:84 , srv1329531800:84 , srv1332481752:84 , srv1366490787:84 , srv1390246974:84 , srv1410418875:84 , srv1420491599:84 , srv1423655457:84 , srv1444811643:84 , srv1457285484:84 , srv1476470663:84 , srv1531953854:84 , srv1540211406:84 , srv1563862214:84 , srv157368198:84 , srv158538720:84 , srv1592294761:84 , srv1601783150:84 , srv1644738360:84 , srv1645700633:84 , srv1653892970:84 , srv1653932599:84 , srv1663906932:84 , srv1693875252:84 , srv1700495364:84 , srv1720159957:84 , srv1741191849:84 , srv1798184562:84 , srv1823985543:84 , srv1832231463:84 , srv1875979806:84 , srv18956697:84 , srv1940514235:84 , srv1984531595:84 , srv2012043025:84 , srv2019971083:84 , srv2024043493:84 , srv2043526671:84 , srv2062301145:84 , srv2114993914:84 , srv227914051:84 , srv271414573:84 , srv293374339:84 , srv29660817:84 , srv322210300:84 , srv337830795:84 , srv371442670:84 , srv384175769:84 , srv393830152:84 , srv394817033:84 , srv411028848:84 , srv417413747:84 , srv418650140:84 , srv429617551:84 , srv435993691:84 , srv449280482:84 , srv473102433:84 , srv48009145:84 , srv488689213:84 , srv513065112:84 , srv536838544:84 , srv574808243:84 , srv581504532:84 , srv592481529:84 , srv619209755:84 , srv620684828:84 , srv651170896:84 , srv667778674:84 , srv680731207:84 , srv695427303:84 , srv715149469:84 , srv716896741:84 , srv725604806:84 , srv738035631:84 , srv743827438:84 , srv754207795:84 , srv759829684:84 , srv812947038:84 , srv838906849:84 , srv920824958:84 , srv968921496:84 , srv973344826:84 , srv992794229:84 , srv998614548:84 , srv1624732650:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-04-22 01:53:52,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:52,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1940514235=48, srv394817033=65, srv620684828=81, srv1178909593=8, srv1420491599=21, srv1134306238=6, srv513065112=75, srv393830152=64, srv754207795=91, srv1476470663=25, srv2043526671=53, srv1563862214=28, srv1274863374=14, srv1003487539=0, srv1832231463=45, srv998614548=99, srv1823985543=44, srv1410418875=20, srv157368198=29, srv1984531595=49, srv574808243=77, srv111111552=4, srv581504532=78, srv293374339=58, srv738035631=89, srv1121319703=5, srv1601783150=32, srv2012043025=50, srv473102433=72, srv992794229=98, srv371442670=62, srv973344826=97, srv2019971083=51, srv271414573=57, srv2024043493=52, srv651170896=82, srv1540211406=27, srv18956697=47, srv2062301145=54, srv743827438=90, srv1663906932=38, srv322210300=60, srv1457285484=24, srv158538720=30, srv1311545670=15, srv536838544=76, srv1366490787=18, srv1653892970=36, srv1645700633=35, srv11915219=9, srv1213489394=12, srv29660817=59, srv418650140=68, srv488689213=74, srv1444811643=23, srv1624732650=33, srv920824958=95, srv1644738360=34, srv227914051=56, srv715149469=86, srv1798184562=43, srv838906849=94, srv411028848=66, srv619209755=80, srv48009145=73, srv716896741=87, srv1653932599=37, srv435993691=70, srv2114993914=55, srv1166573466=7, srv337830795=61, srv109487103=3, srv667778674=83, srv1592294761=31, srv125742280=13, srv1700495364=40, srv680731207=84, srv1329531800=16, srv1423655457=22, srv449280482=71, srv384175769=63, srv1531953854=26, srv968921496=96, srv592481529=79, srv695427303=85, srv429617551=69, srv1092101582=2, srv1211008246=11, srv812947038=93, srv417413747=67, srv725604806=88, srv1004563329=1, srv1741191849=42, srv1875979806=46, srv759829684=92, srv1200549333=10, srv1720159957=41, srv1332481752=17, srv1390246974=19, srv1693875252=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:52,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:52,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:52,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:52,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:52,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:52,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:52,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:52,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:52,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:52,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:52,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:52,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:52,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:52,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:52,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:52,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:52,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:52,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:52,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:52,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:52,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:52,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:52,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:52,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:52,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:52,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:52,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:52,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:52,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:52,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:52,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 01:53:52,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 01:53:52,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 01:53:52,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 01:53:52,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 01:53:52,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 01:53:52,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 01:53:52,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 01:53:52,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 01:53:52,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 01:53:52,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 01:53:52,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 01:53:52,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 01:53:52,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 01:53:53,002 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:53,003 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 01:53:53,003 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 01:53:53,380 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 595 ms to try 5000 different iterations.  Found a solution that moves 4670 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18353276645119848. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040825134505992115, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058257526500353526, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5188888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040825134505992115, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17841666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.249, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:53,392 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1200549333:82 , srv651170896:82 , srv812947038:82 , srv1693875252:83 , srv1940514235:83 , srv293374339:83 , srv620684828:83 , srv968921496:83 , srv1166573466:84 , srv1531953854:84 , srv1592294761:84 , srv1720159957:84 , srv1875979806:84 , srv227914051:84 , srv322210300:84 , srv513065112:84 , srv592481529:84 , srv680731207:84 , srv716896741:84 , srv838906849:84 , srv1092101582:85 , srv109487103:85 , srv1213489394:85 , srv1329531800:85 , srv1741191849:85 , srv1832231463:85 , srv2012043025:85 , srv2024043493:85 , srv394817033:85 , srv429617551:85 , srv581504532:85 , srv715149469:85 , srv743827438:85 , srv754207795:85 , srv973344826:85 , srv1004563329:86 , srv1211008246:86 , srv125742280:86 , srv1274863374:86 , srv1410418875:86 , srv1423655457:86 , srv1457285484:86 , srv1476470663:86 , srv1540211406:86 , srv1563862214:86 , srv158538720:86 , srv1644738360:86 , srv1645700633:86 , srv1663906932:86 , srv1823985543:86 , srv18956697:86 , srv1984531595:86 , srv2019971083:86 , srv2114993914:86 , srv271414573:86 , srv393830152:86 , srv435993691:86 , srv473102433:86 , srv536838544:86 , srv725604806:86 , srv992794229:86 , srv1178909593:87 , srv11915219:87 , srv157368198:87 , srv1601783150:87 , srv1700495364:87 , srv2062301145:87 , srv384175769:87 , srv411028848:87 , srv417413747:87 , srv574808243:87 , srv667778674:87 , srv695427303:87 , srv998614548:87 , srv1003487539:88 , srv111111552:88 , srv1121319703:88 , srv1134306238:88 , srv1311545670:88 , srv1366490787:88 , srv1653892970:88 , srv1798184562:88 , srv2043526671:88 , srv29660817:88 , srv371442670:88 , srv449280482:88 , srv1390246974:89 , srv1653932599:89 , srv337830795:89 , srv619209755:89 , srv920824958:89 , srv1444811643:90 , srv418650140:90 , srv1420491599:91 , srv738035631:91 , srv759829684:91 , srv48009145:93 , srv1332481752:94 , srv488689213:95 , srv1624732650:453 }
2024-04-22 01:53:53,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:53,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1940514235=48, srv394817033=65, srv620684828=81, srv1178909593=8, srv1420491599=21, srv1134306238=6, srv513065112=75, srv393830152=64, srv754207795=91, srv1476470663=25, srv2043526671=53, srv1563862214=28, srv1274863374=14, srv1003487539=0, srv1832231463=45, srv998614548=99, srv1823985543=44, srv1410418875=20, srv157368198=29, srv1984531595=49, srv574808243=77, srv111111552=4, srv581504532=78, srv293374339=58, srv738035631=89, srv1121319703=5, srv1601783150=32, srv2012043025=50, srv473102433=72, srv992794229=98, srv371442670=62, srv973344826=97, srv2019971083=51, srv271414573=57, srv2024043493=52, srv651170896=82, srv1540211406=27, srv18956697=47, srv2062301145=54, srv743827438=90, srv1663906932=38, srv322210300=60, srv1457285484=24, srv158538720=30, srv1311545670=15, srv536838544=76, srv1366490787=18, srv1653892970=36, srv1645700633=35, srv11915219=9, srv1213489394=12, srv29660817=59, srv418650140=68, srv488689213=74, srv1444811643=23, srv1624732650=33, srv920824958=95, srv1644738360=34, srv227914051=56, srv715149469=86, srv1798184562=43, srv838906849=94, srv411028848=66, srv619209755=80, srv48009145=73, srv716896741=87, srv1653932599=37, srv435993691=70, srv2114993914=55, srv1166573466=7, srv337830795=61, srv109487103=3, srv667778674=83, srv1592294761=31, srv125742280=13, srv1700495364=40, srv680731207=84, srv1329531800=16, srv1423655457=22, srv449280482=71, srv384175769=63, srv1531953854=26, srv968921496=96, srv592481529=79, srv695427303=85, srv429617551=69, srv1092101582=2, srv1211008246=11, srv812947038=93, srv417413747=67, srv725604806=88, srv1004563329=1, srv1741191849=42, srv1875979806=46, srv759829684=92, srv1200549333=10, srv1720159957=41, srv1332481752=17, srv1390246974=19, srv1693875252=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:53,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:53,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:53,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 01:53:53,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 01:53:53,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 01:53:53,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 01:53:53,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 01:53:53,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 01:53:53,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 01:53:53,600 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040825134505992115, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058257526500353526, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040825134505992115, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17841666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.249, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:53,600 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 01:53:53,600 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18350006641228653, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040825134505992115, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058257526500353526, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040825134505992115, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17841666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.249, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 01:53:53,859 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 405 ms to try 5000 different iterations.  Found a solution that moves 2669 regions; Going from a computed imbalance of 0.18350006641228653 to a new imbalance of 0.016227869759553946. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022730439983820967); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03827949667014792, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29655555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022730439983820967); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:53,870 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv488689213:76 , srv1332481752:78 , srv738035631:79 , srv48009145:80 , srv759829684:80 , srv1444811643:81 , srv337830795:81 , srv418650140:81 , srv1420491599:82 , srv998614548:82 , srv1003487539:83 , srv1121319703:83 , srv11915219:83 , srv1311545670:83 , srv1663906932:83 , srv1798184562:83 , srv411028848:83 , srv449280482:83 , srv619209755:83 , srv125742280:84 , srv1366490787:84 , srv1563862214:84 , srv158538720:84 , srv371442670:84 , srv384175769:84 , srv417413747:84 , srv667778674:84 , srv1645700633:85 , srv1653932599:85 , srv1700495364:85 , srv1741191849:85 , srv1984531595:85 , srv2019971083:85 , srv2043526671:85 , srv29660817:85 , srv394817033:85 , srv435993691:85 , srv574808243:85 , srv695427303:85 , srv920824958:85 , srv973344826:85 , srv109487103:86 , srv1134306238:86 , srv1178909593:86 , srv1213489394:86 , srv1329531800:86 , srv1410418875:86 , srv1423655457:86 , srv1476470663:86 , srv157368198:86 , srv1601783150:86 , srv1653892970:86 , srv1832231463:86 , srv18956697:86 , srv2012043025:86 , srv2114993914:86 , srv271414573:86 , srv473102433:86 , srv592481529:86 , srv715149469:86 , srv1004563329:87 , srv1092101582:87 , srv111111552:87 , srv1274863374:87 , srv1531953854:87 , srv1644738360:87 , srv1720159957:87 , srv1875979806:87 , srv2062301145:87 , srv227914051:87 , srv513065112:87 , srv536838544:87 , srv725604806:87 , srv838906849:87 , srv1211008246:88 , srv1390246974:88 , srv1457285484:88 , srv1540211406:88 , srv1693875252:88 , srv1823985543:88 , srv1940514235:88 , srv393830152:88 , srv429617551:88 , srv581504532:88 , srv620684828:88 , srv743827438:88 , srv754207795:88 , srv968921496:88 , srv992794229:88 , srv1166573466:89 , srv1200549333:89 , srv1592294761:89 , srv322210300:89 , srv651170896:89 , srv680731207:89 , srv293374339:90 , srv716896741:90 , srv2024043493:91 , srv812947038:91 , srv1624732650:523 }
2024-04-22 01:53:53,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:53,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1940514235=48, srv394817033=65, srv620684828=81, srv1178909593=8, srv1420491599=21, srv1134306238=6, srv513065112=75, srv393830152=64, srv754207795=91, srv1476470663=25, srv2043526671=53, srv1563862214=28, srv1274863374=14, srv1003487539=0, srv1832231463=45, srv998614548=99, srv1823985543=44, srv1410418875=20, srv157368198=29, srv1984531595=49, srv574808243=77, srv111111552=4, srv581504532=78, srv293374339=58, srv738035631=89, srv1121319703=5, srv1601783150=32, srv2012043025=50, srv473102433=72, srv992794229=98, srv371442670=62, srv973344826=97, srv2019971083=51, srv271414573=57, srv2024043493=52, srv651170896=82, srv1540211406=27, srv18956697=47, srv2062301145=54, srv743827438=90, srv1663906932=38, srv322210300=60, srv1457285484=24, srv158538720=30, srv1311545670=15, srv536838544=76, srv1366490787=18, srv1653892970=36, srv1645700633=35, srv11915219=9, srv1213489394=12, srv29660817=59, srv418650140=68, srv488689213=74, srv1444811643=23, srv1624732650=33, srv920824958=95, srv1644738360=34, srv227914051=56, srv715149469=86, srv1798184562=43, srv838906849=94, srv411028848=66, srv619209755=80, srv48009145=73, srv716896741=87, srv1653932599=37, srv435993691=70, srv2114993914=55, srv1166573466=7, srv337830795=61, srv109487103=3, srv667778674=83, srv1592294761=31, srv125742280=13, srv1700495364=40, srv680731207=84, srv1329531800=16, srv1423655457=22, srv449280482=71, srv384175769=63, srv1531953854=26, srv968921496=96, srv592481529=79, srv695427303=85, srv429617551=69, srv1092101582=2, srv1211008246=11, srv812947038=93, srv417413747=67, srv725604806=88, srv1004563329=1, srv1741191849=42, srv1875979806=46, srv759829684=92, srv1200549333=10, srv1720159957=41, srv1332481752=17, srv1390246974=19, srv1693875252=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:53,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:53,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 01:53:53,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 01:53:53,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 01:53:53,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 01:53:53,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 01:53:53,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 01:53:53,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 01:53:53,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 01:53:53,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 01:53:54,034 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022730439983820967); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03827949667014792, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022730439983820967); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:54,034 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 01:53:54,035 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016209181022111552, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022730439983820967); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03827949667014792, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022730439983820967); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013083333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04608333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 01:53:54,280 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 346 ms to try 5000 different iterations.  Found a solution that moves 963 regions; Going from a computed imbalance of 0.016209181022111552 to a new imbalance of 6.781716571828088E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007115458828034776); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011547005383792514); MoveCostFunction : (multiplier=7.0, imbalance=0.107, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007115458828034776); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:54,282 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1390246974:81 , srv111111552:82 , srv2024043493:82 , srv1004563329:83 , srv1540211406:83 , srv1644738360:83 , srv1653932599:83 , srv29660817:83 , srv992794229:83 , srv1134306238:84 , srv1178909593:84 , srv1420491599:84 , srv1457285484:84 , srv157368198:84 , srv1653892970:84 , srv2043526671:84 , srv2062301145:84 , srv293374339:84 , srv393830152:84 , srv429617551:84 , srv48009145:84 , srv536838544:84 , srv619209755:84 , srv680731207:84 , srv725604806:84 , srv743827438:84 , srv920824958:84 , srv1092101582:85 , srv1166573466:85 , srv1211008246:85 , srv1274863374:85 , srv1332481752:85 , srv1366490787:85 , srv1410418875:85 , srv1444811643:85 , srv1531953854:85 , srv1592294761:85 , srv1601783150:85 , srv1700495364:85 , srv2114993914:85 , srv271414573:85 , srv322210300:85 , srv449280482:85 , srv574808243:85 , srv581504532:85 , srv695427303:85 , srv716896741:85 , srv754207795:85 , srv812947038:85 , srv1121319703:86 , srv1200549333:86 , srv1311545670:86 , srv1329531800:86 , srv1476470663:86 , srv1645700633:86 , srv1693875252:86 , srv1720159957:86 , srv1798184562:86 , srv1823985543:86 , srv1875979806:86 , srv18956697:86 , srv1940514235:86 , srv2019971083:86 , srv227914051:86 , srv371442670:86 , srv411028848:86 , srv417413747:86 , srv418650140:86 , srv435993691:86 , srv473102433:86 , srv620684828:86 , srv715149469:86 , srv759829684:86 , srv838906849:86 , srv1003487539:87 , srv109487103:87 , srv11915219:87 , srv1213489394:87 , srv125742280:87 , srv1423655457:87 , srv1563862214:87 , srv1741191849:87 , srv1832231463:87 , srv1984531595:87 , srv2012043025:87 , srv384175769:87 , srv394817033:87 , srv488689213:87 , srv513065112:87 , srv592481529:87 , srv651170896:87 , srv667778674:87 , srv738035631:87 , srv968921496:87 , srv973344826:87 , srv158538720:88 , srv1663906932:88 , srv337830795:88 , srv998614548:89 , srv1624732650:545 }
2024-04-22 01:53:54,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:54,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1940514235=48, srv394817033=65, srv620684828=81, srv1178909593=8, srv1420491599=21, srv1134306238=6, srv513065112=75, srv393830152=64, srv754207795=91, srv1476470663=25, srv2043526671=53, srv1563862214=28, srv1274863374=14, srv1003487539=0, srv1832231463=45, srv998614548=99, srv1823985543=44, srv1410418875=20, srv157368198=29, srv1984531595=49, srv574808243=77, srv111111552=4, srv581504532=78, srv293374339=58, srv738035631=89, srv1121319703=5, srv1601783150=32, srv2012043025=50, srv473102433=72, srv992794229=98, srv371442670=62, srv973344826=97, srv2019971083=51, srv271414573=57, srv2024043493=52, srv651170896=82, srv1540211406=27, srv18956697=47, srv2062301145=54, srv743827438=90, srv1663906932=38, srv322210300=60, srv1457285484=24, srv158538720=30, srv1311545670=15, srv536838544=76, srv1366490787=18, srv1653892970=36, srv1645700633=35, srv11915219=9, srv1213489394=12, srv29660817=59, srv418650140=68, srv488689213=74, srv1444811643=23, srv1624732650=33, srv920824958=95, srv1644738360=34, srv227914051=56, srv715149469=86, srv1798184562=43, srv838906849=94, srv411028848=66, srv619209755=80, srv48009145=73, srv716896741=87, srv1653932599=37, srv435993691=70, srv2114993914=55, srv1166573466=7, srv337830795=61, srv109487103=3, srv667778674=83, srv1592294761=31, srv125742280=13, srv1700495364=40, srv680731207=84, srv1329531800=16, srv1423655457=22, srv449280482=71, srv384175769=63, srv1531953854=26, srv968921496=96, srv592481529=79, srv695427303=85, srv429617551=69, srv1092101582=2, srv1211008246=11, srv812947038=93, srv417413747=67, srv725604806=88, srv1004563329=1, srv1741191849=42, srv1875979806=46, srv759829684=92, srv1200549333=10, srv1720159957=41, srv1332481752=17, srv1390246974=19, srv1693875252=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:54,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:54,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 01:53:54,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 01:53:54,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 01:53:54,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 01:53:54,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 01:53:54,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 01:53:54,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 01:53:54,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 01:53:54,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 01:53:54,453 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007115458828034776); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011547005383792514); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007115458828034776); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:54,454 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 01:53:54,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.714285870602813E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007115458828034776); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011547005383792514); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007115458828034776); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 01:53:54,853 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 524 ms to try 5000 different iterations.  Found a solution that moves 255 regions; Going from a computed imbalance of 6.714285870602813E-4 to a new imbalance of 2.8845187346704413E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.028333333333333332, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:54,855 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1823985543:82 , srv1211008246:83 , srv1003487539:84 , srv109487103:84 , srv1134306238:84 , srv1166573466:84 , srv1213489394:84 , srv1274863374:84 , srv1390246974:84 , srv1420491599:84 , srv1423655457:84 , srv1457285484:84 , srv1476470663:84 , srv1563862214:84 , srv157368198:84 , srv158538720:84 , srv1592294761:84 , srv1601783150:84 , srv1653932599:84 , srv1720159957:84 , srv1832231463:84 , srv18956697:84 , srv1984531595:84 , srv2012043025:84 , srv2024043493:84 , srv2062301145:84 , srv322210300:84 , srv337830795:84 , srv371442670:84 , srv384175769:84 , srv393830152:84 , srv394817033:84 , srv473102433:84 , srv488689213:84 , srv513065112:84 , srv581504532:84 , srv620684828:84 , srv651170896:84 , srv667778674:84 , srv716896741:84 , srv754207795:84 , srv812947038:84 , srv920824958:84 , srv968921496:84 , srv973344826:84 , srv998614548:84 , srv1004563329:85 , srv1092101582:85 , srv111111552:85 , srv1121319703:85 , srv11915219:85 , srv1200549333:85 , srv125742280:85 , srv1311545670:85 , srv1329531800:85 , srv1332481752:85 , srv1366490787:85 , srv1410418875:85 , srv1645700633:85 , srv1653892970:85 , srv1693875252:85 , srv1700495364:85 , srv1741191849:85 , srv1798184562:85 , srv1875979806:85 , srv1940514235:85 , srv2019971083:85 , srv2043526671:85 , srv2114993914:85 , srv227914051:85 , srv271414573:85 , srv293374339:85 , srv429617551:85 , srv435993691:85 , srv48009145:85 , srv536838544:85 , srv574808243:85 , srv592481529:85 , srv680731207:85 , srv695427303:85 , srv715149469:85 , srv725604806:85 , srv738035631:85 , srv743827438:85 , srv759829684:85 , srv838906849:85 , srv992794229:85 , srv1178909593:86 , srv1444811643:86 , srv1531953854:86 , srv1540211406:86 , srv1644738360:86 , srv1663906932:86 , srv411028848:86 , srv417413747:86 , srv418650140:86 , srv449280482:86 , srv619209755:86 , srv29660817:87 , srv1624732650:621 }
2024-04-22 01:53:54,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:54,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1940514235=48, srv394817033=65, srv620684828=81, srv1178909593=8, srv1420491599=21, srv1134306238=6, srv513065112=75, srv393830152=64, srv754207795=91, srv1476470663=25, srv2043526671=53, srv1563862214=28, srv1274863374=14, srv1003487539=0, srv1832231463=45, srv998614548=99, srv1823985543=44, srv1410418875=20, srv157368198=29, srv1984531595=49, srv574808243=77, srv111111552=4, srv581504532=78, srv293374339=58, srv738035631=89, srv1121319703=5, srv1601783150=32, srv2012043025=50, srv473102433=72, srv992794229=98, srv371442670=62, srv973344826=97, srv2019971083=51, srv271414573=57, srv2024043493=52, srv651170896=82, srv1540211406=27, srv18956697=47, srv2062301145=54, srv743827438=90, srv1663906932=38, srv322210300=60, srv1457285484=24, srv158538720=30, srv1311545670=15, srv536838544=76, srv1366490787=18, srv1653892970=36, srv1645700633=35, srv11915219=9, srv1213489394=12, srv29660817=59, srv418650140=68, srv488689213=74, srv1444811643=23, srv1624732650=33, srv920824958=95, srv1644738360=34, srv227914051=56, srv715149469=86, srv1798184562=43, srv838906849=94, srv411028848=66, srv619209755=80, srv48009145=73, srv716896741=87, srv1653932599=37, srv435993691=70, srv2114993914=55, srv1166573466=7, srv337830795=61, srv109487103=3, srv667778674=83, srv1592294761=31, srv125742280=13, srv1700495364=40, srv680731207=84, srv1329531800=16, srv1423655457=22, srv449280482=71, srv384175769=63, srv1531953854=26, srv968921496=96, srv592481529=79, srv695427303=85, srv429617551=69, srv1092101582=2, srv1211008246=11, srv812947038=93, srv417413747=67, srv725604806=88, srv1004563329=1, srv1741191849=42, srv1875979806=46, srv759829684=92, srv1200549333=10, srv1720159957=41, srv1332481752=17, srv1390246974=19, srv1693875252=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:54,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:54,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:54,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:54,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 01:53:54,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 01:53:54,882 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 01:53:54,883 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 01:53:54,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 01:53:54,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 01:53:54,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 01:53:54,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 01:53:54,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 01:53:54,996 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:54,996 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-04-22 01:53:54,996 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.7059639183418287E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-04-22 01:53:55,261 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 361 ms to try 5000 different iterations.  Found a solution that moves 29 regions; Going from a computed imbalance of 2.7059639183418287E-5 to a new imbalance of 8.462826326129825E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0032222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:55,261 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1178909593:83 , srv1423655457:83 , srv1540211406:83 , srv1653892970:83 , srv1663906932:83 , srv29660817:83 , srv417413747:83 , srv418650140:83 , srv716896741:83 , srv1003487539:84 , srv1092101582:84 , srv109487103:84 , srv111111552:84 , srv1121319703:84 , srv1134306238:84 , srv1166573466:84 , srv11915219:84 , srv1200549333:84 , srv1211008246:84 , srv1213489394:84 , srv125742280:84 , srv1274863374:84 , srv1311545670:84 , srv1329531800:84 , srv1332481752:84 , srv1366490787:84 , srv1390246974:84 , srv1410418875:84 , srv1444811643:84 , srv1457285484:84 , srv1476470663:84 , srv1531953854:84 , srv158538720:84 , srv1592294761:84 , srv1601783150:84 , srv1644738360:84 , srv1645700633:84 , srv1693875252:84 , srv1700495364:84 , srv1741191849:84 , srv1798184562:84 , srv1823985543:84 , srv1832231463:84 , srv1875979806:84 , srv18956697:84 , srv1940514235:84 , srv1984531595:84 , srv2012043025:84 , srv2019971083:84 , srv2024043493:84 , srv2043526671:84 , srv2062301145:84 , srv2114993914:84 , srv227914051:84 , srv271414573:84 , srv293374339:84 , srv322210300:84 , srv337830795:84 , srv371442670:84 , srv384175769:84 , srv393830152:84 , srv411028848:84 , srv429617551:84 , srv435993691:84 , srv449280482:84 , srv473102433:84 , srv48009145:84 , srv488689213:84 , srv513065112:84 , srv536838544:84 , srv574808243:84 , srv581504532:84 , srv592481529:84 , srv619209755:84 , srv651170896:84 , srv667778674:84 , srv680731207:84 , srv695427303:84 , srv715149469:84 , srv725604806:84 , srv738035631:84 , srv743827438:84 , srv754207795:84 , srv759829684:84 , srv812947038:84 , srv838906849:84 , srv920824958:84 , srv968921496:84 , srv992794229:84 , srv998614548:84 , srv1004563329:85 , srv1420491599:85 , srv1563862214:85 , srv157368198:85 , srv1653932599:85 , srv1720159957:85 , srv394817033:85 , srv620684828:85 , srv973344826:85 , srv1624732650:684 }
2024-04-22 01:53:55,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:55,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1940514235=48, srv394817033=65, srv620684828=81, srv1178909593=8, srv1420491599=21, srv1134306238=6, srv513065112=75, srv393830152=64, srv754207795=91, srv1476470663=25, srv2043526671=53, srv1563862214=28, srv1274863374=14, srv1003487539=0, srv1832231463=45, srv998614548=99, srv1823985543=44, srv1410418875=20, srv157368198=29, srv1984531595=49, srv574808243=77, srv111111552=4, srv581504532=78, srv293374339=58, srv738035631=89, srv1121319703=5, srv1601783150=32, srv2012043025=50, srv473102433=72, srv992794229=98, srv371442670=62, srv973344826=97, srv2019971083=51, srv271414573=57, srv2024043493=52, srv651170896=82, srv1540211406=27, srv18956697=47, srv2062301145=54, srv743827438=90, srv1663906932=38, srv322210300=60, srv1457285484=24, srv158538720=30, srv1311545670=15, srv536838544=76, srv1366490787=18, srv1653892970=36, srv1645700633=35, srv11915219=9, srv1213489394=12, srv29660817=59, srv418650140=68, srv488689213=74, srv1444811643=23, srv1624732650=33, srv920824958=95, srv1644738360=34, srv227914051=56, srv715149469=86, srv1798184562=43, srv838906849=94, srv411028848=66, srv619209755=80, srv48009145=73, srv716896741=87, srv1653932599=37, srv435993691=70, srv2114993914=55, srv1166573466=7, srv337830795=61, srv109487103=3, srv667778674=83, srv1592294761=31, srv125742280=13, srv1700495364=40, srv680731207=84, srv1329531800=16, srv1423655457=22, srv449280482=71, srv384175769=63, srv1531953854=26, srv968921496=96, srv592481529=79, srv695427303=85, srv429617551=69, srv1092101582=2, srv1211008246=11, srv812947038=93, srv417413747=67, srv725604806=88, srv1004563329=1, srv1741191849=42, srv1875979806=46, srv759829684=92, srv1200549333=10, srv1720159957=41, srv1332481752=17, srv1390246974=19, srv1693875252=39} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:55,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-04-22 01:53:55,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-04-22 01:53:55,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-04-22 01:53:55,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-04-22 01:53:55,280 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-04-22 01:53:55,410 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:55,411 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1178909593:83 , srv1423655457:83 , srv1540211406:83 , srv1653892970:83 , srv1663906932:83 , srv29660817:83 , srv417413747:83 , srv418650140:83 , srv716896741:83 , srv1003487539:84 , srv1092101582:84 , srv109487103:84 , srv111111552:84 , srv1121319703:84 , srv1134306238:84 , srv1166573466:84 , srv11915219:84 , srv1200549333:84 , srv1211008246:84 , srv1213489394:84 , srv125742280:84 , srv1274863374:84 , srv1311545670:84 , srv1329531800:84 , srv1332481752:84 , srv1366490787:84 , srv1390246974:84 , srv1410418875:84 , srv1444811643:84 , srv1457285484:84 , srv1476470663:84 , srv1531953854:84 , srv158538720:84 , srv1592294761:84 , srv1601783150:84 , srv1644738360:84 , srv1645700633:84 , srv1693875252:84 , srv1700495364:84 , srv1741191849:84 , srv1798184562:84 , srv1823985543:84 , srv1832231463:84 , srv1875979806:84 , srv18956697:84 , srv1940514235:84 , srv1984531595:84 , srv2012043025:84 , srv2019971083:84 , srv2024043493:84 , srv2043526671:84 , srv2062301145:84 , srv2114993914:84 , srv227914051:84 , srv271414573:84 , srv293374339:84 , srv322210300:84 , srv337830795:84 , srv371442670:84 , srv384175769:84 , srv393830152:84 , srv411028848:84 , srv429617551:84 , srv435993691:84 , srv449280482:84 , srv473102433:84 , srv48009145:84 , srv488689213:84 , srv513065112:84 , srv536838544:84 , srv574808243:84 , srv581504532:84 , srv592481529:84 , srv619209755:84 , srv651170896:84 , srv667778674:84 , srv680731207:84 , srv695427303:84 , srv715149469:84 , srv725604806:84 , srv738035631:84 , srv743827438:84 , srv754207795:84 , srv759829684:84 , srv812947038:84 , srv838906849:84 , srv920824958:84 , srv968921496:84 , srv992794229:84 , srv998614548:84 , srv1004563329:85 , srv1420491599:85 , srv1563862214:85 , srv157368198:85 , srv1653932599:85 , srv1720159957:85 , srv394817033:85 , srv620684828:85 , srv973344826:85 , srv1624732650:684 }
2024-04-22 01:53:55,448 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=244 (was 248), ProcessCount=168 (was 168), AvailableMemoryMB=12031 (was 12544)
2024-04-22 01:53:55,459 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=244, ProcessCount=168, AvailableMemoryMB=12029
2024-04-22 01:53:55,460 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-04-22 01:53:55,461 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-04-22 01:53:55,480 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004833790:84 , srv1033214139:84 , srv1197314974:84 , srv126405232:84 , srv1301889105:84 , srv137960761:84 , srv1386332657:84 , srv1426919823:84 , srv1453980946:84 , srv1537366872:84 , srv1565421361:84 , srv1573000490:84 , srv158160349:84 , srv160776809:84 , srv1612379234:84 , srv1748334142:84 , srv2082798266:84 , srv283303220:84 , srv303054611:84 , srv355830612:84 , srv366436811:84 , srv402743520:84 , srv435077173:84 , srv448441164:84 , srv497606300:84 , srv654856420:84 , srv675472233:84 , srv829326143:84 , srv977624738:84 , srv508627536:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-04-22 01:53:55,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:53:55,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1197314974=2, srv355830612=19, srv1612379234=14, srv160776809=13, srv1033214139=1, srv435077173=22, srv126405232=3, srv1386332657=6, srv1748334142=15, srv448441164=23, srv497606300=24, srv654856420=26, srv829326143=28, srv508627536=25, srv1453980946=8, srv137960761=5, srv1426919823=7, srv303054611=18, srv1004833790=0, srv366436811=20, srv158160349=12, srv2082798266=16, srv1301889105=4, srv1565421361=10, srv1573000490=11, srv283303220=17, srv675472233=27, srv977624738=29, srv1537366872=9, srv402743520=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:53:55,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:53:55,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:53:55,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:53:55,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:53:55,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:53:55,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:53:55,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:53:55,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-22 01:53:55,521 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1063647128663496, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:53:55,521 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912528195001394, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1063647128663496, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-04-22 01:55:55,505 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6310980 different iterations.  Found a solution that moves 1851 regions; Going from a computed imbalance of 0.9912528195001394 to a new imbalance of 4.320326340186438E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6855555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:55:55,513 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1004833790:90 , srv1033214139:90 , srv1197314974:90 , srv126405232:90 , srv1301889105:90 , srv137960761:90 , srv1386332657:90 , srv1426919823:90 , srv1453980946:90 , srv1537366872:90 , srv1565421361:90 , srv1573000490:90 , srv158160349:90 , srv160776809:90 , srv1612379234:90 , srv1748334142:90 , srv2082798266:90 , srv283303220:90 , srv303054611:90 , srv355830612:90 , srv366436811:90 , srv402743520:90 , srv435077173:90 , srv448441164:90 , srv497606300:90 , srv508627536:90 , srv654856420:90 , srv675472233:90 , srv829326143:90 , srv977624738:90 }
2024-04-22 01:55:55,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-04-22 01:55:55,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1197314974=2, srv355830612=19, srv1612379234=14, srv160776809=13, srv1033214139=1, srv435077173=22, srv126405232=3, srv1386332657=6, srv1748334142=15, srv448441164=23, srv497606300=24, srv654856420=26, srv829326143=28, srv508627536=25, srv1453980946=8, srv137960761=5, srv1426919823=7, srv303054611=18, srv1004833790=0, srv366436811=20, srv158160349=12, srv2082798266=16, srv1301889105=4, srv1565421361=10, srv1573000490=11, srv283303220=17, srv675472233=27, srv977624738=29, srv1537366872=9, srv402743520=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-04-22 01:55:55,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-04-22 01:55:55,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-04-22 01:55:55,536 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-04-22 01:55:55,536 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1004833790:90 , srv1033214139:90 , srv1197314974:90 , srv126405232:90 , srv1301889105:90 , srv137960761:90 , srv1386332657:90 , srv1426919823:90 , srv1453980946:90 , srv1537366872:90 , srv1565421361:90 , srv1573000490:90 , srv158160349:90 , srv160776809:90 , srv1612379234:90 , srv1748334142:90 , srv2082798266:90 , srv283303220:90 , srv303054611:90 , srv355830612:90 , srv366436811:90 , srv402743520:90 , srv435077173:90 , srv448441164:90 , srv497606300:90 , srv508627536:90 , srv654856420:90 , srv675472233:90 , srv829326143:90 , srv977624738:90 }
2024-04-22 01:55:55,556 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=289 (was 289), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=185 (was 244), ProcessCount=168 (was 168), AvailableMemoryMB=12176 (was 12029) - AvailableMemoryMB LEAK? -