2023-08-23 16:55:51,222 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-08-23 16:55:51,472 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-08-23 16:55:51,570 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-08-23 16:55:51,623 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-23 16:55:51,629 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-23 16:55:51,639 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=169, AvailableMemoryMB=9594
2023-08-23 16:55:51,641 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-23 16:55:51,641 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-08-23 16:55:51,767 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100697522:84 , srv1020242684:84 , srv1208576620:84 , srv1217234665:84 , srv1236234480:84 , srv1236263841:84 , srv1276101494:84 , srv1278556295:84 , srv1290023425:84 , srv1307942130:84 , srv1342151387:84 , srv134621620:84 , srv1348840073:84 , srv134986378:84 , srv1367868946:84 , srv1373003438:84 , srv1381821167:84 , srv1392209411:84 , srv1424895141:84 , srv1477775264:84 , srv1483740369:84 , srv1492603964:84 , srv1515044264:84 , srv1515113085:84 , srv1586202538:84 , srv1590181476:84 , srv1613793682:84 , srv1626965866:84 , srv1645067075:84 , srv1645533001:84 , srv1665923065:84 , srv1668471509:84 , srv1679435901:84 , srv1681645075:84 , srv1690880490:84 , srv1695404647:84 , srv170383744:84 , srv1706980275:84 , srv170763270:84 , srv1717070764:84 , srv1720439811:84 , srv1728544075:84 , srv1736166267:84 , srv1744033381:84 , srv1759059788:84 , srv1766763518:84 , srv1813512854:84 , srv1826504444:84 , srv1840259432:84 , srv1841347281:84 , srv1877779209:84 , srv1878864580:84 , srv1904430892:84 , srv1920916717:84 , srv1928464768:84 , srv1972524663:84 , srv1977629420:84 , srv1981218620:84 , srv2013014701:84 , srv2034649517:84 , srv2054698550:84 , srv2071091839:84 , srv2098635713:84 , srv2106643676:84 , srv2109829610:84 , srv2118145427:84 , srv2126500927:84 , srv231444911:84 , srv30289190:84 , srv332898869:84 , srv342526215:84 , srv362303264:84 , srv441239977:84 , srv457352783:84 , srv472239858:84 , srv49122665:84 , srv495143267:84 , srv505891331:84 , srv506888386:84 , srv534167107:84 , srv550317380:84 , srv595402905:84 , srv599789018:84 , srv611686712:84 , srv612270768:84 , srv621391442:84 , srv651879276:84 , srv66068632:84 , srv66566889:84 , srv678389752:84 , srv715250259:84 , srv799580966:84 , srv81612283:84 , srv817987213:84 , srv825001647:84 , srv83818548:84 , srv839566407:84 , srv873391203:84 , srv957775171:84 , srv123975054:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-08-23 16:55:51,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:55:51,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1515044264=23, srv170763270=39, srv2071091839=62, srv599789018=83, srv1276101494=7, srv1981218620=58, srv2109829610=65, srv1977629420=57, srv49122665=76, srv134986378=14, srv1217234665=3, srv1348840073=13, srv83818548=96, srv1477775264=20, srv1626965866=28, srv612270768=85, srv1841347281=50, srv457352783=74, srv1020242684=1, srv1367868946=15, srv1717070764=40, srv342526215=71, srv2054698550=61, srv1483740369=21, srv1877779209=51, srv1920916717=54, srv534167107=80, srv1492603964=22, srv1681645075=34, srv1679435901=33, srv1645533001=30, srv1290023425=9, srv362303264=72, srv825001647=95, srv1373003438=16, srv957775171=99, srv1392209411=18, srv1424895141=19, srv1668471509=32, srv621391442=86, srv2034649517=60, srv1695404647=36, srv2013014701=59, srv715250259=91, srv2126500927=67, srv1307942130=10, srv1878864580=52, srv839566407=97, srv1665923065=31, srv1720439811=41, srv1342151387=11, srv1515113085=24, srv1728544075=42, srv472239858=75, srv1586202538=25, srv678389752=90, srv2106643676=64, srv30289190=69, srv1613793682=27, srv66566889=89, srv81612283=93, srv1972524663=56, srv1766763518=46, srv1813512854=47, srv1826504444=48, srv66068632=88, srv1381821167=17, srv170383744=37, srv100697522=0, srv231444911=68, srv1690880490=35, srv1590181476=26, srv595402905=82, srv134621620=12, srv873391203=98, srv505891331=78, srv651879276=87, srv332898869=70, srv1706980275=38, srv1236234480=4, srv1904430892=53, srv2098635713=63, srv1744033381=44, srv611686712=84, srv1759059788=45, srv1645067075=29, srv799580966=92, srv817987213=94, srv1736166267=43, srv123975054=6, srv506888386=79, srv550317380=81, srv1278556295=8, srv1208576620=2, srv441239977=73, srv2118145427=66, srv495143267=77, srv1928464768=55, srv1236263841=5, srv1840259432=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:55:51,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:55:51,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:55:51,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:55:51,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:55:51,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:55:51,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:55:51,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:55:51,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:55:51,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:55:51,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:55:51,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:55:51,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:55:51,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 16:55:51,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 16:55:51,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 16:55:51,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 16:55:51,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 16:55:51,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 16:55:51,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 16:55:51,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 16:55:51,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 16:55:51,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 16:55:51,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 16:55:51,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 16:55:51,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 16:55:51,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 16:55:51,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 16:55:51,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 16:55:51,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 16:55:51,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 16:55:52,316 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:52,316 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 16:55:52,317 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 16:55:52,647 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 716 ms to try 5000 different iterations.  Found a solution that moves 4630 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18845215088492714. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03961656853900696, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.050807177131148656, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5144444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03961656853900696, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18416666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24658333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:52,659 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1348840073:82 , srv170383744:82 , srv1736166267:82 , srv1759059788:82 , srv1977629420:82 , srv441239977:82 , srv1373003438:83 , srv1515113085:83 , srv1645067075:83 , srv1720439811:83 , srv1877779209:83 , srv1878864580:83 , srv2098635713:83 , srv342526215:83 , srv825001647:83 , srv134621620:84 , srv1665923065:84 , srv1706980275:84 , srv2109829610:84 , srv472239858:84 , srv621391442:84 , srv81612283:84 , srv839566407:84 , srv957775171:84 , srv1367868946:85 , srv1492603964:85 , srv1613793682:85 , srv1690880490:85 , srv1744033381:85 , srv1766763518:85 , srv1813512854:85 , srv2034649517:85 , srv2118145427:85 , srv534167107:85 , srv612270768:85 , srv651879276:85 , srv66068632:85 , srv66566889:85 , srv799580966:85 , srv1236234480:86 , srv1276101494:86 , srv1290023425:86 , srv1342151387:86 , srv1392209411:86 , srv1586202538:86 , srv1717070764:86 , srv1920916717:86 , srv2071091839:86 , srv495143267:86 , srv505891331:86 , srv550317380:86 , srv873391203:86 , srv1217234665:87 , srv1236263841:87 , srv1307942130:87 , srv1477775264:87 , srv1626965866:87 , srv1679435901:87 , srv1695404647:87 , srv1728544075:87 , srv1826504444:87 , srv1904430892:87 , srv49122665:87 , srv599789018:87 , srv611686712:87 , srv83818548:87 , srv100697522:88 , srv1020242684:88 , srv1381821167:88 , srv1515044264:88 , srv1645533001:88 , srv1668471509:88 , srv1681645075:88 , srv1840259432:88 , srv1928464768:88 , srv332898869:88 , srv506888386:88 , srv1424895141:89 , srv1972524663:89 , srv1981218620:89 , srv2013014701:89 , srv2054698550:89 , srv2106643676:89 , srv2126500927:89 , srv362303264:89 , srv678389752:89 , srv715250259:89 , srv1278556295:90 , srv1483740369:90 , srv170763270:90 , srv1841347281:90 , srv30289190:90 , srv817987213:90 , srv231444911:91 , srv457352783:91 , srv134986378:92 , srv1590181476:93 , srv595402905:93 , srv1208576620:94 , srv123975054:442 }
2023-08-23 16:55:52,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:55:52,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1515044264=23, srv170763270=39, srv2071091839=62, srv599789018=83, srv1276101494=7, srv1981218620=58, srv2109829610=65, srv1977629420=57, srv49122665=76, srv134986378=14, srv1217234665=3, srv1348840073=13, srv83818548=96, srv1477775264=20, srv1626965866=28, srv612270768=85, srv1841347281=50, srv457352783=74, srv1020242684=1, srv1367868946=15, srv1717070764=40, srv342526215=71, srv2054698550=61, srv1483740369=21, srv1877779209=51, srv1920916717=54, srv534167107=80, srv1492603964=22, srv1681645075=34, srv1679435901=33, srv1645533001=30, srv1290023425=9, srv362303264=72, srv825001647=95, srv1373003438=16, srv957775171=99, srv1392209411=18, srv1424895141=19, srv1668471509=32, srv621391442=86, srv2034649517=60, srv1695404647=36, srv2013014701=59, srv715250259=91, srv2126500927=67, srv1307942130=10, srv1878864580=52, srv839566407=97, srv1665923065=31, srv1720439811=41, srv1342151387=11, srv1515113085=24, srv1728544075=42, srv472239858=75, srv1586202538=25, srv678389752=90, srv2106643676=64, srv30289190=69, srv1613793682=27, srv66566889=89, srv81612283=93, srv1972524663=56, srv1766763518=46, srv1813512854=47, srv1826504444=48, srv66068632=88, srv1381821167=17, srv170383744=37, srv100697522=0, srv231444911=68, srv1690880490=35, srv1590181476=26, srv595402905=82, srv134621620=12, srv873391203=98, srv505891331=78, srv651879276=87, srv332898869=70, srv1706980275=38, srv1236234480=4, srv1904430892=53, srv2098635713=63, srv1744033381=44, srv611686712=84, srv1759059788=45, srv1645067075=29, srv799580966=92, srv817987213=94, srv1736166267=43, srv123975054=6, srv506888386=79, srv550317380=81, srv1278556295=8, srv1208576620=2, srv441239977=73, srv2118145427=66, srv495143267=77, srv1928464768=55, srv1236263841=5, srv1840259432=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:55:52,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:55:52,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:55:52,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:55:52,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:55:52,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 16:55:52,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 16:55:52,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 16:55:52,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 16:55:52,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 16:55:52,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 16:55:52,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 16:55:52,818 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03961656853900696, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.050807177131148656, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03961656853900696, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18416666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24658333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:52,818 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 16:55:52,819 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1884197309320016, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03961656853900696, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.050807177131148656, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03961656853900696, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18416666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24658333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 16:55:53,061 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 351 ms to try 5000 different iterations.  Found a solution that moves 2641 regions; Going from a computed imbalance of 0.1884197309320016 to a new imbalance of 0.014789266764198162. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021936433918460514); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0312478955520364, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29344444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021936433918460514); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:53,072 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1208576620:79 , srv595402905:79 , srv1590181476:80 , srv170763270:80 , srv30289190:80 , srv1981218620:81 , srv2126500927:81 , srv1278556295:82 , srv1483740369:82 , srv457352783:82 , srv506888386:82 , srv678389752:82 , srv817987213:82 , srv100697522:83 , srv1020242684:83 , srv134986378:83 , srv1424895141:83 , srv1477775264:83 , srv1515044264:83 , srv1679435901:83 , srv1841347281:83 , srv1972524663:83 , srv2054698550:83 , srv2106643676:83 , srv231444911:83 , srv362303264:83 , srv599789018:83 , srv1613793682:84 , srv1668471509:84 , srv1840259432:84 , srv2013014701:84 , srv611686712:84 , srv1236234480:85 , srv1276101494:85 , srv1342151387:85 , srv1381821167:85 , srv1626965866:85 , srv1681645075:85 , srv1695404647:85 , srv1728544075:85 , srv1813512854:85 , srv1920916717:85 , srv1928464768:85 , srv332898869:85 , srv49122665:85 , srv66068632:85 , srv715250259:85 , srv873391203:85 , srv1236263841:86 , srv1290023425:86 , srv1392209411:86 , srv1586202538:86 , srv1645533001:86 , srv1744033381:86 , srv1766763518:86 , srv1826504444:86 , srv1904430892:86 , srv2071091839:86 , srv342526215:86 , srv550317380:86 , srv621391442:86 , srv1217234665:87 , srv1307942130:87 , srv134621620:87 , srv1373003438:87 , srv1492603964:87 , srv1706980275:87 , srv1717070764:87 , srv495143267:87 , srv651879276:87 , srv81612283:87 , srv83818548:87 , srv839566407:87 , srv1367868946:88 , srv1690880490:88 , srv170383744:88 , srv2034649517:88 , srv2118145427:88 , srv472239858:88 , srv612270768:88 , srv957775171:88 , srv1348840073:89 , srv1515113085:89 , srv1665923065:89 , srv1720439811:89 , srv1877779209:89 , srv1878864580:89 , srv2109829610:89 , srv505891331:89 , srv66566889:89 , srv799580966:89 , srv825001647:89 , srv1736166267:90 , srv1759059788:90 , srv441239977:90 , srv534167107:90 , srv1645067075:91 , srv2098635713:91 , srv1977629420:92 , srv123975054:527 }
2023-08-23 16:55:53,078 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:55:53,080 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1515044264=23, srv170763270=39, srv2071091839=62, srv599789018=83, srv1276101494=7, srv1981218620=58, srv2109829610=65, srv1977629420=57, srv49122665=76, srv134986378=14, srv1217234665=3, srv1348840073=13, srv83818548=96, srv1477775264=20, srv1626965866=28, srv612270768=85, srv1841347281=50, srv457352783=74, srv1020242684=1, srv1367868946=15, srv1717070764=40, srv342526215=71, srv2054698550=61, srv1483740369=21, srv1877779209=51, srv1920916717=54, srv534167107=80, srv1492603964=22, srv1681645075=34, srv1679435901=33, srv1645533001=30, srv1290023425=9, srv362303264=72, srv825001647=95, srv1373003438=16, srv957775171=99, srv1392209411=18, srv1424895141=19, srv1668471509=32, srv621391442=86, srv2034649517=60, srv1695404647=36, srv2013014701=59, srv715250259=91, srv2126500927=67, srv1307942130=10, srv1878864580=52, srv839566407=97, srv1665923065=31, srv1720439811=41, srv1342151387=11, srv1515113085=24, srv1728544075=42, srv472239858=75, srv1586202538=25, srv678389752=90, srv2106643676=64, srv30289190=69, srv1613793682=27, srv66566889=89, srv81612283=93, srv1972524663=56, srv1766763518=46, srv1813512854=47, srv1826504444=48, srv66068632=88, srv1381821167=17, srv170383744=37, srv100697522=0, srv231444911=68, srv1690880490=35, srv1590181476=26, srv595402905=82, srv134621620=12, srv873391203=98, srv505891331=78, srv651879276=87, srv332898869=70, srv1706980275=38, srv1236234480=4, srv1904430892=53, srv2098635713=63, srv1744033381=44, srv611686712=84, srv1759059788=45, srv1645067075=29, srv799580966=92, srv817987213=94, srv1736166267=43, srv123975054=6, srv506888386=79, srv550317380=81, srv1278556295=8, srv1208576620=2, srv441239977=73, srv2118145427=66, srv495143267=77, srv1928464768=55, srv1236263841=5, srv1840259432=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:55:53,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:55:53,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:55:53,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:55:53,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:55:53,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:55:53,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:55:53,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:55:53,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:55:53,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:55:53,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 16:55:53,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 16:55:53,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 16:55:53,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 16:55:53,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 16:55:53,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 16:55:53,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 16:55:53,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 16:55:53,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 16:55:53,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 16:55:53,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 16:55:53,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 16:55:53,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 16:55:53,217 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021936433918460514); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0312478955520364, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021936433918460514); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:53,217 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 16:55:53,217 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014770774086946246, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021936433918460514); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0312478955520364, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021936433918460514); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 16:55:53,431 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 297 ms to try 5000 different iterations.  Found a solution that moves 976 regions; Going from a computed imbalance of 0.014770774086946246 to a new imbalance of 4.960366121250971E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006568677540342965); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006649810339649611); MoveCostFunction : (multiplier=7.0, imbalance=0.10844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006568677540342965); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:53,432 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv134986378:82 , srv505891331:82 , srv799580966:82 , srv1645533001:83 , srv1208576620:84 , srv1217234665:84 , srv1307942130:84 , srv1424895141:84 , srv1586202538:84 , srv1590181476:84 , srv1645067075:84 , srv1665923065:84 , srv1690880490:84 , srv1826504444:84 , srv1841347281:84 , srv1878864580:84 , srv1928464768:84 , srv1977629420:84 , srv2013014701:84 , srv2034649517:84 , srv2098635713:84 , srv2118145427:84 , srv231444911:84 , srv457352783:84 , srv534167107:84 , srv612270768:84 , srv715250259:84 , srv83818548:84 , srv957775171:84 , srv1236234480:85 , srv1278556295:85 , srv1367868946:85 , srv1381821167:85 , srv1392209411:85 , srv1483740369:85 , srv1515113085:85 , srv1695404647:85 , srv1717070764:85 , srv1728544075:85 , srv1736166267:85 , srv1759059788:85 , srv1877779209:85 , srv1972524663:85 , srv2054698550:85 , srv2071091839:85 , srv2109829610:85 , srv332898869:85 , srv362303264:85 , srv495143267:85 , srv595402905:85 , srv651879276:85 , srv66566889:85 , srv825001647:85 , srv100697522:86 , srv1020242684:86 , srv1236263841:86 , srv1276101494:86 , srv1290023425:86 , srv1342151387:86 , srv134621620:86 , srv1373003438:86 , srv1515044264:86 , srv1626965866:86 , srv1668471509:86 , srv1679435901:86 , srv1681645075:86 , srv1706980275:86 , srv1720439811:86 , srv1744033381:86 , srv1904430892:86 , srv1920916717:86 , srv2106643676:86 , srv441239977:86 , srv49122665:86 , srv678389752:86 , srv81612283:86 , srv817987213:86 , srv839566407:86 , srv1348840073:87 , srv1492603964:87 , srv170383744:87 , srv170763270:87 , srv1766763518:87 , srv1813512854:87 , srv1840259432:87 , srv1981218620:87 , srv2126500927:87 , srv472239858:87 , srv506888386:87 , srv550317380:87 , srv599789018:87 , srv621391442:87 , srv873391203:87 , srv1613793682:88 , srv30289190:88 , srv342526215:88 , srv611686712:88 , srv66068632:88 , srv1477775264:89 , srv123975054:547 }
2023-08-23 16:55:53,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:55:53,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1515044264=23, srv170763270=39, srv2071091839=62, srv599789018=83, srv1276101494=7, srv1981218620=58, srv2109829610=65, srv1977629420=57, srv49122665=76, srv134986378=14, srv1217234665=3, srv1348840073=13, srv83818548=96, srv1477775264=20, srv1626965866=28, srv612270768=85, srv1841347281=50, srv457352783=74, srv1020242684=1, srv1367868946=15, srv1717070764=40, srv342526215=71, srv2054698550=61, srv1483740369=21, srv1877779209=51, srv1920916717=54, srv534167107=80, srv1492603964=22, srv1681645075=34, srv1679435901=33, srv1645533001=30, srv1290023425=9, srv362303264=72, srv825001647=95, srv1373003438=16, srv957775171=99, srv1392209411=18, srv1424895141=19, srv1668471509=32, srv621391442=86, srv2034649517=60, srv1695404647=36, srv2013014701=59, srv715250259=91, srv2126500927=67, srv1307942130=10, srv1878864580=52, srv839566407=97, srv1665923065=31, srv1720439811=41, srv1342151387=11, srv1515113085=24, srv1728544075=42, srv472239858=75, srv1586202538=25, srv678389752=90, srv2106643676=64, srv30289190=69, srv1613793682=27, srv66566889=89, srv81612283=93, srv1972524663=56, srv1766763518=46, srv1813512854=47, srv1826504444=48, srv66068632=88, srv1381821167=17, srv170383744=37, srv100697522=0, srv231444911=68, srv1690880490=35, srv1590181476=26, srv595402905=82, srv134621620=12, srv873391203=98, srv505891331=78, srv651879276=87, srv332898869=70, srv1706980275=38, srv1236234480=4, srv1904430892=53, srv2098635713=63, srv1744033381=44, srv611686712=84, srv1759059788=45, srv1645067075=29, srv799580966=92, srv817987213=94, srv1736166267=43, srv123975054=6, srv506888386=79, srv550317380=81, srv1278556295=8, srv1208576620=2, srv441239977=73, srv2118145427=66, srv495143267=77, srv1928464768=55, srv1236263841=5, srv1840259432=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:55:53,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:55:53,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:55:53,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:55:53,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:55:53,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:55:53,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:55:53,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:55:53,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 16:55:53,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 16:55:53,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 16:55:53,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 16:55:53,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 16:55:53,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 16:55:53,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 16:55:53,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 16:55:53,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 16:55:53,573 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006568677540342965); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006649810339649611); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006568677540342965); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:53,573 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-08-23 16:55:53,573 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.892025140569901E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006568677540342965); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006649810339649611); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006568677540342965); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0030833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-08-23 16:55:53,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 379 ms to try 5000 different iterations.  Found a solution that moves 210 regions; Going from a computed imbalance of 4.892025140569901E-4 to a new imbalance of 1.0721411850924163E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.023333333333333334); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:53,850 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1236263841:83 , srv1477775264:83 , srv1492603964:83 , srv1645067075:83 , srv1668471509:83 , srv1681645075:83 , srv1840259432:83 , srv1904430892:83 , srv534167107:83 , srv550317380:83 , srv611686712:83 , srv66566889:83 , srv1217234665:84 , srv1290023425:84 , srv1307942130:84 , srv1348840073:84 , srv1367868946:84 , srv1381821167:84 , srv1392209411:84 , srv1626965866:84 , srv1717070764:84 , srv1720439811:84 , srv1766763518:84 , srv1826504444:84 , srv1977629420:84 , srv2098635713:84 , srv2106643676:84 , srv2109829610:84 , srv231444911:84 , srv30289190:84 , srv441239977:84 , srv472239858:84 , srv49122665:84 , srv495143267:84 , srv66068632:84 , srv715250259:84 , srv817987213:84 , srv83818548:84 , srv100697522:85 , srv1020242684:85 , srv1208576620:85 , srv1276101494:85 , srv1278556295:85 , srv134621620:85 , srv134986378:85 , srv1424895141:85 , srv1483740369:85 , srv1515044264:85 , srv1515113085:85 , srv1590181476:85 , srv1613793682:85 , srv1645533001:85 , srv1665923065:85 , srv1679435901:85 , srv1690880490:85 , srv1695404647:85 , srv170383744:85 , srv1706980275:85 , srv170763270:85 , srv1728544075:85 , srv1736166267:85 , srv1759059788:85 , srv1813512854:85 , srv1841347281:85 , srv1877779209:85 , srv1920916717:85 , srv1928464768:85 , srv1972524663:85 , srv1981218620:85 , srv2013014701:85 , srv2034649517:85 , srv2054698550:85 , srv2071091839:85 , srv2118145427:85 , srv2126500927:85 , srv332898869:85 , srv342526215:85 , srv362303264:85 , srv457352783:85 , srv505891331:85 , srv506888386:85 , srv595402905:85 , srv599789018:85 , srv612270768:85 , srv621391442:85 , srv651879276:85 , srv678389752:85 , srv81612283:85 , srv839566407:85 , srv873391203:85 , srv1236234480:86 , srv1342151387:86 , srv1373003438:86 , srv1586202538:86 , srv1744033381:86 , srv1878864580:86 , srv799580966:86 , srv825001647:86 , srv957775171:86 , srv123975054:626 }
2023-08-23 16:55:53,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:55:53,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1515044264=23, srv170763270=39, srv2071091839=62, srv599789018=83, srv1276101494=7, srv1981218620=58, srv2109829610=65, srv1977629420=57, srv49122665=76, srv134986378=14, srv1217234665=3, srv1348840073=13, srv83818548=96, srv1477775264=20, srv1626965866=28, srv612270768=85, srv1841347281=50, srv457352783=74, srv1020242684=1, srv1367868946=15, srv1717070764=40, srv342526215=71, srv2054698550=61, srv1483740369=21, srv1877779209=51, srv1920916717=54, srv534167107=80, srv1492603964=22, srv1681645075=34, srv1679435901=33, srv1645533001=30, srv1290023425=9, srv362303264=72, srv825001647=95, srv1373003438=16, srv957775171=99, srv1392209411=18, srv1424895141=19, srv1668471509=32, srv621391442=86, srv2034649517=60, srv1695404647=36, srv2013014701=59, srv715250259=91, srv2126500927=67, srv1307942130=10, srv1878864580=52, srv839566407=97, srv1665923065=31, srv1720439811=41, srv1342151387=11, srv1515113085=24, srv1728544075=42, srv472239858=75, srv1586202538=25, srv678389752=90, srv2106643676=64, srv30289190=69, srv1613793682=27, srv66566889=89, srv81612283=93, srv1972524663=56, srv1766763518=46, srv1813512854=47, srv1826504444=48, srv66068632=88, srv1381821167=17, srv170383744=37, srv100697522=0, srv231444911=68, srv1690880490=35, srv1590181476=26, srv595402905=82, srv134621620=12, srv873391203=98, srv505891331=78, srv651879276=87, srv332898869=70, srv1706980275=38, srv1236234480=4, srv1904430892=53, srv2098635713=63, srv1744033381=44, srv611686712=84, srv1759059788=45, srv1645067075=29, srv799580966=92, srv817987213=94, srv1736166267=43, srv123975054=6, srv506888386=79, srv550317380=81, srv1278556295=8, srv1208576620=2, srv441239977=73, srv2118145427=66, srv495143267=77, srv1928464768=55, srv1236263841=5, srv1840259432=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:55:53,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:55:53,866 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:55:53,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-08-23 16:55:53,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-08-23 16:55:53,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-08-23 16:55:53,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-08-23 16:55:53,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-08-23 16:55:53,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-08-23 16:55:53,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-08-23 16:55:53,959 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.250960422335587E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:53,960 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1236263841:83 , srv1477775264:83 , srv1492603964:83 , srv1645067075:83 , srv1668471509:83 , srv1681645075:83 , srv1840259432:83 , srv1904430892:83 , srv534167107:83 , srv550317380:83 , srv611686712:83 , srv66566889:83 , srv1217234665:84 , srv1290023425:84 , srv1307942130:84 , srv1348840073:84 , srv1367868946:84 , srv1381821167:84 , srv1392209411:84 , srv1626965866:84 , srv1717070764:84 , srv1720439811:84 , srv1766763518:84 , srv1826504444:84 , srv1977629420:84 , srv2098635713:84 , srv2106643676:84 , srv2109829610:84 , srv231444911:84 , srv30289190:84 , srv441239977:84 , srv472239858:84 , srv49122665:84 , srv495143267:84 , srv66068632:84 , srv715250259:84 , srv817987213:84 , srv83818548:84 , srv100697522:85 , srv1020242684:85 , srv1208576620:85 , srv1276101494:85 , srv1278556295:85 , srv134621620:85 , srv134986378:85 , srv1424895141:85 , srv1483740369:85 , srv1515044264:85 , srv1515113085:85 , srv1590181476:85 , srv1613793682:85 , srv1645533001:85 , srv1665923065:85 , srv1679435901:85 , srv1690880490:85 , srv1695404647:85 , srv170383744:85 , srv1706980275:85 , srv170763270:85 , srv1728544075:85 , srv1736166267:85 , srv1759059788:85 , srv1813512854:85 , srv1841347281:85 , srv1877779209:85 , srv1920916717:85 , srv1928464768:85 , srv1972524663:85 , srv1981218620:85 , srv2013014701:85 , srv2034649517:85 , srv2054698550:85 , srv2071091839:85 , srv2118145427:85 , srv2126500927:85 , srv332898869:85 , srv342526215:85 , srv362303264:85 , srv457352783:85 , srv505891331:85 , srv506888386:85 , srv595402905:85 , srv599789018:85 , srv612270768:85 , srv621391442:85 , srv651879276:85 , srv678389752:85 , srv81612283:85 , srv839566407:85 , srv873391203:85 , srv1236234480:86 , srv1342151387:86 , srv1373003438:86 , srv1586202538:86 , srv1744033381:86 , srv1878864580:86 , srv799580966:86 , srv825001647:86 , srv957775171:86 , srv123975054:626 }
2023-08-23 16:55:53,982 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=293 (was 293), ProcessCount=169 (was 169), AvailableMemoryMB=9007 (was 9594)
2023-08-23 16:55:53,990 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=293, ProcessCount=169, AvailableMemoryMB=9006
2023-08-23 16:55:53,991 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-08-23 16:55:53,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-08-23 16:55:54,003 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1189086795:84 , srv1206044195:84 , srv127181575:84 , srv1343185476:84 , srv1383462553:84 , srv1399711476:84 , srv141585471:84 , srv1442699648:84 , srv1442989264:84 , srv1493786429:84 , srv1530041479:84 , srv1783357448:84 , srv1823169577:84 , srv1861515346:84 , srv1872679087:84 , srv208532610:84 , srv295518703:84 , srv394379177:84 , srv434326429:84 , srv486492418:84 , srv517513463:84 , srv531786550:84 , srv605853442:84 , srv70963557:84 , srv816181912:84 , srv865151398:84 , srv957982125:84 , srv979172514:84 , srv999411704:84 , srv1614429497:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-08-23 16:55:54,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:55:54,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1343185476=3, srv434326429=19, srv865151398=26, srv1399711476=5, srv1383462553=4, srv517513463=21, srv979172514=28, srv999411704=29, srv1206044195=1, srv1442699648=7, srv1783357448=12, srv531786550=22, srv1872679087=15, srv1189086795=0, srv127181575=2, srv816181912=25, srv1861515346=14, srv1614429497=11, srv394379177=18, srv605853442=23, srv957982125=27, srv486492418=20, srv208532610=16, srv70963557=24, srv1442989264=8, srv1823169577=13, srv295518703=17, srv141585471=6, srv1493786429=9, srv1530041479=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:55:54,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:55:54,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:55:54,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:55:54,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:55:54,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:55:54,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:55:54,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:55:54,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-23 16:55:54,046 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1106772756725671, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:55:54,046 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912541783745978, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1106772756725671, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-08-23 16:57:54,025 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9223547 different iterations.  Found a solution that moves 1846 regions; Going from a computed imbalance of 0.9912541783745978 to a new imbalance of 4.308656090753194E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6837037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:57:54,030 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1189086795:90 , srv1206044195:90 , srv127181575:90 , srv1343185476:90 , srv1383462553:90 , srv1399711476:90 , srv141585471:90 , srv1442699648:90 , srv1442989264:90 , srv1493786429:90 , srv1530041479:90 , srv1614429497:90 , srv1783357448:90 , srv1823169577:90 , srv1861515346:90 , srv1872679087:90 , srv208532610:90 , srv295518703:90 , srv394379177:90 , srv434326429:90 , srv486492418:90 , srv517513463:90 , srv531786550:90 , srv605853442:90 , srv70963557:90 , srv816181912:90 , srv865151398:90 , srv957982125:90 , srv979172514:90 , srv999411704:90 }
2023-08-23 16:57:54,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-08-23 16:57:54,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1343185476=3, srv434326429=19, srv865151398=26, srv1399711476=5, srv1383462553=4, srv517513463=21, srv979172514=28, srv999411704=29, srv1206044195=1, srv1442699648=7, srv1783357448=12, srv531786550=22, srv1872679087=15, srv1189086795=0, srv127181575=2, srv816181912=25, srv1861515346=14, srv1614429497=11, srv394379177=18, srv605853442=23, srv957982125=27, srv486492418=20, srv208532610=16, srv70963557=24, srv1442989264=8, srv1823169577=13, srv295518703=17, srv141585471=6, srv1493786429=9, srv1530041479=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-08-23 16:57:54,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-08-23 16:57:54,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-08-23 16:57:54,045 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-08-23 16:57:54,045 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1189086795:90 , srv1206044195:90 , srv127181575:90 , srv1343185476:90 , srv1383462553:90 , srv1399711476:90 , srv141585471:90 , srv1442699648:90 , srv1442989264:90 , srv1493786429:90 , srv1530041479:90 , srv1614429497:90 , srv1783357448:90 , srv1823169577:90 , srv1861515346:90 , srv1872679087:90 , srv208532610:90 , srv295518703:90 , srv394379177:90 , srv434326429:90 , srv486492418:90 , srv517513463:90 , srv531786550:90 , srv605853442:90 , srv70963557:90 , srv816181912:90 , srv865151398:90 , srv957982125:90 , srv979172514:90 , srv999411704:90 }
2023-08-23 16:57:54,055 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=156 (was 293), ProcessCount=170 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=8468 (was 9006)