2023-11-19 05:55:15,155 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-19 05:55:15,429 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-19 05:55:15,526 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-19 05:55:15,590 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-19 05:55:15,598 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-19 05:55:15,611 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=134, ProcessCount=168, AvailableMemoryMB=18438
2023-11-19 05:55:15,613 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-19 05:55:15,614 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-19 05:55:15,739 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006768162:84 , srv1010680475:84 , srv1057583115:84 , srv1078978856:84 , srv1088451917:84 , srv1103983883:84 , srv1116510611:84 , srv1131905923:84 , srv113884954:84 , srv1149463047:84 , srv1191824607:84 , srv1193919042:84 , srv1196590772:84 , srv1232716529:84 , srv1248887491:84 , srv1251150576:84 , srv1339285259:84 , srv1394095056:84 , srv1396821334:84 , srv1427788070:84 , srv1441417436:84 , srv1446552449:84 , srv1454213918:84 , srv1489140336:84 , srv1499265910:84 , srv1508278538:84 , srv1519151996:84 , srv1531854253:84 , srv1538549932:84 , srv1558235626:84 , srv1610207569:84 , srv1622234:84 , srv1654052347:84 , srv1660337771:84 , srv1677135203:84 , srv1699102806:84 , srv1720750751:84 , srv1738714834:84 , srv1739100857:84 , srv1752854577:84 , srv1868829410:84 , srv1875881637:84 , srv1879791268:84 , srv1887558160:84 , srv1893735558:84 , srv1948185052:84 , srv1954464639:84 , srv2013791664:84 , srv2035246134:84 , srv2040059558:84 , srv2107064987:84 , srv2112263648:84 , srv2127642011:84 , srv2134777287:84 , srv248059999:84 , srv259232564:84 , srv282373973:84 , srv283946079:84 , srv340196:84 , srv34152148:84 , srv348394566:84 , srv363719516:84 , srv372182474:84 , srv379473650:84 , srv419315501:84 , srv43158002:84 , srv477898885:84 , srv499650706:84 , srv511202588:84 , srv547443555:84 , srv569706771:84 , srv569932667:84 , srv633484942:84 , srv667177954:84 , srv681060864:84 , srv681314057:84 , srv682203349:84 , srv686098898:84 , srv691060902:84 , srv701612762:84 , srv750618927:84 , srv788940162:84 , srv813378783:84 , srv818651601:84 , srv825648330:84 , srv826397409:84 , srv844861984:84 , srv871829116:84 , srv881326790:84 , srv883287252:84 , srv899390246:84 , srv912753136:84 , srv933305187:84 , srv944214994:84 , srv946474033:84 , srv947047334:84 , srv96979101:84 , srv974230228:84 , srv983381455:84 , srv1192186039:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-19 05:55:15,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:15,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1010680475=1, srv2112263648=52, srv1088451917=4, srv2013791664=48, srv912753136=92, srv1739100857=39, srv363719516=62, srv691060902=79, srv511202588=69, srv681060864=75, srv379473650=64, srv1394095056=18, srv1875881637=42, srv701612762=80, srv788940162=82, srv1339285259=17, srv1248887491=15, srv259232564=56, srv826397409=86, srv818651601=84, srv499650706=68, srv1893735558=45, srv1193919042=12, srv825648330=85, srv1531854253=28, srv1006768162=0, srv2040059558=50, srv569706771=71, srv750618927=81, srv1454213918=23, srv1191824607=10, srv844861984=87, srv569932667=72, srv633484942=73, srv1446552449=22, srv1078978856=3, srv1879791268=43, srv1057583115=2, srv947047334=96, srv1149463047=9, srv340196=59, srv43158002=66, srv477898885=67, srv1954464639=47, srv1699102806=36, srv348394566=61, srv881326790=89, srv1519151996=27, srv2134777287=54, srv248059999=55, srv1677135203=35, srv34152148=60, srv1192186039=11, srv282373973=57, srv1489140336=24, srv682203349=77, srv1948185052=46, srv1196590772=13, srv1116510611=6, srv1752854577=40, srv944214994=94, srv1396821334=19, srv96979101=97, srv1251150576=16, srv1499265910=25, srv1538549932=29, srv686098898=78, srv1622234=32, srv1887558160=44, srv2035246134=49, srv1610207569=31, srv899390246=91, srv1441417436=21, srv1868829410=41, srv283946079=58, srv372182474=63, srv883287252=90, srv1427788070=20, srv1738714834=38, srv933305187=93, srv681314057=76, srv1103983883=5, srv1131905923=7, srv2127642011=53, srv1660337771=34, srv983381455=99, srv419315501=65, srv1232716529=14, srv974230228=98, srv871829116=88, srv667177954=74, srv946474033=95, srv1654052347=33, srv113884954=8, srv547443555=70, srv1508278538=26, srv1558235626=30, srv813378783=83, srv1720750751=37, srv2107064987=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:15,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:15,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:15,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 05:55:15,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 05:55:15,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 05:55:15,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 05:55:15,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 05:55:15,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 05:55:15,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 05:55:15,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 05:55:15,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 05:55:15,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 05:55:16,000 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:16,000 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 05:55:16,000 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 05:55:16,708 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 850 ms to try 5000 different iterations.  Found a solution that moves 4692 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17558573377164297. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039037235095556184, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056337925419991344, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5213333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039037235095556184, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17125, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23258333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:16,730 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1538549932:82 , srv1660337771:82 , srv871829116:82 , srv933305187:82 , srv1010680475:83 , srv1508278538:83 , srv1739100857:83 , srv1948185052:83 , srv259232564:83 , srv569932667:83 , srv686098898:83 , srv750618927:83 , srv1006768162:84 , srv1752854577:84 , srv1893735558:84 , srv248059999:84 , srv511202588:84 , srv633484942:84 , srv691060902:84 , srv818651601:84 , srv825648330:84 , srv1131905923:85 , srv1196590772:85 , srv1427788070:85 , srv1441417436:85 , srv1868829410:85 , srv283946079:85 , srv419315501:85 , srv547443555:85 , srv681314057:85 , srv701612762:85 , srv899390246:85 , srv1057583115:86 , srv1078978856:86 , srv1103983883:86 , srv1116510611:86 , srv1232716529:86 , srv1339285259:86 , srv1446552449:86 , srv1610207569:86 , srv1622234:86 , srv1654052347:86 , srv1720750751:86 , srv1875881637:86 , srv2035246134:86 , srv340196:86 , srv379473650:86 , srv477898885:86 , srv667177954:86 , srv681060864:86 , srv881326790:86 , srv883287252:86 , srv946474033:86 , srv96979101:86 , srv1251150576:87 , srv1394095056:87 , srv1499265910:87 , srv1558235626:87 , srv2013791664:87 , srv2107064987:87 , srv34152148:87 , srv363719516:87 , srv43158002:87 , srv499650706:87 , srv788940162:87 , srv813378783:87 , srv912753136:87 , srv1149463047:88 , srv1193919042:88 , srv1248887491:88 , srv1396821334:88 , srv1489140336:88 , srv1677135203:88 , srv1738714834:88 , srv1887558160:88 , srv682203349:88 , srv944214994:88 , srv974230228:88 , srv1088451917:89 , srv1454213918:89 , srv1531854253:89 , srv1699102806:89 , srv1879791268:89 , srv1954464639:89 , srv2040059558:89 , srv2127642011:89 , srv826397409:89 , srv983381455:89 , srv113884954:90 , srv1519151996:90 , srv2112263648:90 , srv2134777287:90 , srv282373973:90 , srv348394566:90 , srv844861984:90 , srv947047334:91 , srv1191824607:92 , srv372182474:93 , srv569706771:93 , srv1192186039:437 }
2023-11-19 05:55:16,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:16,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1010680475=1, srv2112263648=52, srv1088451917=4, srv2013791664=48, srv912753136=92, srv1739100857=39, srv363719516=62, srv691060902=79, srv511202588=69, srv681060864=75, srv379473650=64, srv1394095056=18, srv1875881637=42, srv701612762=80, srv788940162=82, srv1339285259=17, srv1248887491=15, srv259232564=56, srv826397409=86, srv818651601=84, srv499650706=68, srv1893735558=45, srv1193919042=12, srv825648330=85, srv1531854253=28, srv1006768162=0, srv2040059558=50, srv569706771=71, srv750618927=81, srv1454213918=23, srv1191824607=10, srv844861984=87, srv569932667=72, srv633484942=73, srv1446552449=22, srv1078978856=3, srv1879791268=43, srv1057583115=2, srv947047334=96, srv1149463047=9, srv340196=59, srv43158002=66, srv477898885=67, srv1954464639=47, srv1699102806=36, srv348394566=61, srv881326790=89, srv1519151996=27, srv2134777287=54, srv248059999=55, srv1677135203=35, srv34152148=60, srv1192186039=11, srv282373973=57, srv1489140336=24, srv682203349=77, srv1948185052=46, srv1196590772=13, srv1116510611=6, srv1752854577=40, srv944214994=94, srv1396821334=19, srv96979101=97, srv1251150576=16, srv1499265910=25, srv1538549932=29, srv686098898=78, srv1622234=32, srv1887558160=44, srv2035246134=49, srv1610207569=31, srv899390246=91, srv1441417436=21, srv1868829410=41, srv283946079=58, srv372182474=63, srv883287252=90, srv1427788070=20, srv1738714834=38, srv933305187=93, srv681314057=76, srv1103983883=5, srv1131905923=7, srv2127642011=53, srv1660337771=34, srv983381455=99, srv419315501=65, srv1232716529=14, srv974230228=98, srv871829116=88, srv667177954=74, srv946474033=95, srv1654052347=33, srv113884954=8, srv547443555=70, srv1508278538=26, srv1558235626=30, srv813378783=83, srv1720750751=37, srv2107064987=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:16,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:16,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:16,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:16,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:16,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:16,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 05:55:16,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 05:55:16,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 05:55:16,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 05:55:16,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 05:55:16,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 05:55:16,934 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039037235095556184, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056337925419991344, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039037235095556184, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17125, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23258333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:16,935 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 05:55:16,935 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17555287968543848, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039037235095556184, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056337925419991344, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039037235095556184, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17125, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23258333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 05:55:17,262 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 486 ms to try 5000 different iterations.  Found a solution that moves 2609 regions; Going from a computed imbalance of 0.17555287968543848 to a new imbalance of 0.013874475289518663. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019745228515601138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03208434451159157, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.28988888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019745228515601138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:17,272 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1191824607:79 , srv372182474:79 , srv2127642011:81 , srv826397409:81 , srv844861984:81 , srv947047334:81 , srv113884954:82 , srv1193919042:82 , srv1519151996:82 , srv1699102806:82 , srv1738714834:82 , srv1879791268:82 , srv2134777287:82 , srv1088451917:83 , srv1149463047:83 , srv1248887491:83 , srv1954464639:83 , srv2040059558:83 , srv2112263648:83 , srv348394566:83 , srv974230228:83 , srv1396821334:84 , srv1489140336:84 , srv1531854253:84 , srv1558235626:84 , srv1677135203:84 , srv2013791664:84 , srv43158002:84 , srv569706771:84 , srv881326790:84 , srv944214994:84 , srv1057583115:85 , srv1232716529:85 , srv1394095056:85 , srv1446552449:85 , srv1610207569:85 , srv283946079:85 , srv34152148:85 , srv363719516:85 , srv379473650:85 , srv499650706:85 , srv667177954:85 , srv883287252:85 , srv912753136:85 , srv946474033:85 , srv96979101:85 , srv1006768162:86 , srv1131905923:86 , srv1251150576:86 , srv1339285259:86 , srv1441417436:86 , srv1499265910:86 , srv1622234:86 , srv1887558160:86 , srv1893735558:86 , srv2035246134:86 , srv2107064987:86 , srv248059999:86 , srv282373973:86 , srv477898885:86 , srv511202588:86 , srv681060864:86 , srv682203349:86 , srv788940162:86 , srv813378783:86 , srv825648330:86 , srv1103983883:87 , srv1116510611:87 , srv1427788070:87 , srv1654052347:87 , srv1720750751:87 , srv1868829410:87 , srv1875881637:87 , srv547443555:87 , srv701612762:87 , srv983381455:87 , srv1010680475:88 , srv1538549932:88 , srv1739100857:88 , srv340196:88 , srv681314057:88 , srv691060902:88 , srv818651601:88 , srv1078978856:89 , srv1196590772:89 , srv1508278538:89 , srv1660337771:89 , srv1752854577:89 , srv259232564:89 , srv569932667:89 , srv1948185052:90 , srv899390246:90 , srv750618927:91 , srv871829116:91 , srv1454213918:92 , srv419315501:92 , srv633484942:92 , srv933305187:92 , srv686098898:94 , srv1192186039:512 }
2023-11-19 05:55:17,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:17,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1010680475=1, srv2112263648=52, srv1088451917=4, srv2013791664=48, srv912753136=92, srv1739100857=39, srv363719516=62, srv691060902=79, srv511202588=69, srv681060864=75, srv379473650=64, srv1394095056=18, srv1875881637=42, srv701612762=80, srv788940162=82, srv1339285259=17, srv1248887491=15, srv259232564=56, srv826397409=86, srv818651601=84, srv499650706=68, srv1893735558=45, srv1193919042=12, srv825648330=85, srv1531854253=28, srv1006768162=0, srv2040059558=50, srv569706771=71, srv750618927=81, srv1454213918=23, srv1191824607=10, srv844861984=87, srv569932667=72, srv633484942=73, srv1446552449=22, srv1078978856=3, srv1879791268=43, srv1057583115=2, srv947047334=96, srv1149463047=9, srv340196=59, srv43158002=66, srv477898885=67, srv1954464639=47, srv1699102806=36, srv348394566=61, srv881326790=89, srv1519151996=27, srv2134777287=54, srv248059999=55, srv1677135203=35, srv34152148=60, srv1192186039=11, srv282373973=57, srv1489140336=24, srv682203349=77, srv1948185052=46, srv1196590772=13, srv1116510611=6, srv1752854577=40, srv944214994=94, srv1396821334=19, srv96979101=97, srv1251150576=16, srv1499265910=25, srv1538549932=29, srv686098898=78, srv1622234=32, srv1887558160=44, srv2035246134=49, srv1610207569=31, srv899390246=91, srv1441417436=21, srv1868829410=41, srv283946079=58, srv372182474=63, srv883287252=90, srv1427788070=20, srv1738714834=38, srv933305187=93, srv681314057=76, srv1103983883=5, srv1131905923=7, srv2127642011=53, srv1660337771=34, srv983381455=99, srv419315501=65, srv1232716529=14, srv974230228=98, srv871829116=88, srv667177954=74, srv946474033=95, srv1654052347=33, srv113884954=8, srv547443555=70, srv1508278538=26, srv1558235626=30, srv813378783=83, srv1720750751=37, srv2107064987=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:17,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:17,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:17,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:17,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:17,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:17,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:17,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:17,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 05:55:17,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 05:55:17,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 05:55:17,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 05:55:17,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 05:55:17,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 05:55:17,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 05:55:17,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 05:55:17,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 05:55:17,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 05:55:17,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 05:55:17,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 05:55:17,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 05:55:17,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 05:55:17,436 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019745228515601138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03208434451159157, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019745228515601138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:17,437 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 05:55:17,437 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013856206681055866, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019745228515601138); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03208434451159157, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019745228515601138); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 05:55:17,757 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 419 ms to try 5000 different iterations.  Found a solution that moves 979 regions; Going from a computed imbalance of 0.013856206681055866 to a new imbalance of 9.717259808293251E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005988724492745198); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00794198944903163); MoveCostFunction : (multiplier=7.0, imbalance=0.10877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005988724492745198); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:17,759 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1454213918:78 , srv569706771:80 , srv686098898:80 , srv419315501:81 , srv1078978856:82 , srv282373973:82 , srv633484942:82 , srv899390246:82 , srv983381455:82 , srv2107064987:83 , srv340196:83 , srv1103983883:84 , srv1196590772:84 , srv1251150576:84 , srv1499265910:84 , srv1508278538:84 , srv1622234:84 , srv1752854577:84 , srv1887558160:84 , srv1948185052:84 , srv682203349:84 , srv750618927:84 , srv813378783:84 , srv871829116:84 , srv933305187:84 , srv1088451917:85 , srv1116510611:85 , srv1427788070:85 , srv1489140336:85 , srv1519151996:85 , srv1531854253:85 , srv1654052347:85 , srv1677135203:85 , srv1720750751:85 , srv1875881637:85 , srv1954464639:85 , srv2035246134:85 , srv2112263648:85 , srv259232564:85 , srv34152148:85 , srv348394566:85 , srv363719516:85 , srv379473650:85 , srv477898885:85 , srv499650706:85 , srv547443555:85 , srv681060864:85 , srv681314057:85 , srv691060902:85 , srv701612762:85 , srv788940162:85 , srv818651601:85 , srv944214994:85 , srv947047334:85 , srv113884954:86 , srv1149463047:86 , srv1232716529:86 , srv1248887491:86 , srv1339285259:86 , srv1394095056:86 , srv1396821334:86 , srv1441417436:86 , srv1610207569:86 , srv1660337771:86 , srv1739100857:86 , srv1868829410:86 , srv1879791268:86 , srv2013791664:86 , srv2040059558:86 , srv372182474:86 , srv569932667:86 , srv667177954:86 , srv844861984:86 , srv912753136:86 , srv1006768162:87 , srv1010680475:87 , srv1057583115:87 , srv1131905923:87 , srv1191824607:87 , srv1193919042:87 , srv1446552449:87 , srv1538549932:87 , srv1558235626:87 , srv1699102806:87 , srv1893735558:87 , srv2134777287:87 , srv248059999:87 , srv43158002:87 , srv881326790:87 , srv883287252:87 , srv946474033:87 , srv974230228:87 , srv1738714834:88 , srv2127642011:88 , srv283946079:88 , srv511202588:88 , srv825648330:88 , srv826397409:88 , srv96979101:88 , srv1192186039:562 }
2023-11-19 05:55:17,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:17,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1010680475=1, srv2112263648=52, srv1088451917=4, srv2013791664=48, srv912753136=92, srv1739100857=39, srv363719516=62, srv691060902=79, srv511202588=69, srv681060864=75, srv379473650=64, srv1394095056=18, srv1875881637=42, srv701612762=80, srv788940162=82, srv1339285259=17, srv1248887491=15, srv259232564=56, srv826397409=86, srv818651601=84, srv499650706=68, srv1893735558=45, srv1193919042=12, srv825648330=85, srv1531854253=28, srv1006768162=0, srv2040059558=50, srv569706771=71, srv750618927=81, srv1454213918=23, srv1191824607=10, srv844861984=87, srv569932667=72, srv633484942=73, srv1446552449=22, srv1078978856=3, srv1879791268=43, srv1057583115=2, srv947047334=96, srv1149463047=9, srv340196=59, srv43158002=66, srv477898885=67, srv1954464639=47, srv1699102806=36, srv348394566=61, srv881326790=89, srv1519151996=27, srv2134777287=54, srv248059999=55, srv1677135203=35, srv34152148=60, srv1192186039=11, srv282373973=57, srv1489140336=24, srv682203349=77, srv1948185052=46, srv1196590772=13, srv1116510611=6, srv1752854577=40, srv944214994=94, srv1396821334=19, srv96979101=97, srv1251150576=16, srv1499265910=25, srv1538549932=29, srv686098898=78, srv1622234=32, srv1887558160=44, srv2035246134=49, srv1610207569=31, srv899390246=91, srv1441417436=21, srv1868829410=41, srv283946079=58, srv372182474=63, srv883287252=90, srv1427788070=20, srv1738714834=38, srv933305187=93, srv681314057=76, srv1103983883=5, srv1131905923=7, srv2127642011=53, srv1660337771=34, srv983381455=99, srv419315501=65, srv1232716529=14, srv974230228=98, srv871829116=88, srv667177954=74, srv946474033=95, srv1654052347=33, srv113884954=8, srv547443555=70, srv1508278538=26, srv1558235626=30, srv813378783=83, srv1720750751=37, srv2107064987=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:17,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:17,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 05:55:17,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 05:55:17,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 05:55:17,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 05:55:17,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 05:55:17,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 05:55:17,883 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005988724492745198); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00794198944903163); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005988724492745198); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:17,883 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 05:55:17,883 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=9.648708763122382E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005988724492745198); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00794198944903163); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005988724492745198); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.833333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 05:55:18,131 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 343 ms to try 5000 different iterations.  Found a solution that moves 214 regions; Going from a computed imbalance of 9.648708763122382E-4 to a new imbalance of 4.848859998818074E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.023777777777777776); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:18,132 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1454213918:83 , srv2134777287:83 , srv946474033:83 , srv96979101:83 , srv983381455:83 , srv1010680475:84 , srv1057583115:84 , srv1116510611:84 , srv1131905923:84 , srv113884954:84 , srv1191824607:84 , srv1196590772:84 , srv1248887491:84 , srv1339285259:84 , srv1394095056:84 , srv1396821334:84 , srv1531854253:84 , srv1558235626:84 , srv1654052347:84 , srv1677135203:84 , srv1699102806:84 , srv1720750751:84 , srv1868829410:84 , srv1875881637:84 , srv1887558160:84 , srv2040059558:84 , srv2112263648:84 , srv2127642011:84 , srv259232564:84 , srv283946079:84 , srv348394566:84 , srv372182474:84 , srv419315501:84 , srv43158002:84 , srv511202588:84 , srv547443555:84 , srv569706771:84 , srv569932667:84 , srv681314057:84 , srv750618927:84 , srv788940162:84 , srv825648330:84 , srv826397409:84 , srv883287252:84 , srv912753136:84 , srv933305187:84 , srv974230228:84 , srv1006768162:85 , srv1078978856:85 , srv1103983883:85 , srv1149463047:85 , srv1232716529:85 , srv1251150576:85 , srv1427788070:85 , srv1441417436:85 , srv1446552449:85 , srv1489140336:85 , srv1499265910:85 , srv1519151996:85 , srv1538549932:85 , srv1610207569:85 , srv1660337771:85 , srv1738714834:85 , srv1739100857:85 , srv1752854577:85 , srv1879791268:85 , srv1948185052:85 , srv1954464639:85 , srv2013791664:85 , srv2035246134:85 , srv2107064987:85 , srv248059999:85 , srv282373973:85 , srv340196:85 , srv34152148:85 , srv363719516:85 , srv477898885:85 , srv499650706:85 , srv667177954:85 , srv681060864:85 , srv682203349:85 , srv686098898:85 , srv691060902:85 , srv701612762:85 , srv813378783:85 , srv818651601:85 , srv844861984:85 , srv871829116:85 , srv881326790:85 , srv899390246:85 , srv944214994:85 , srv947047334:85 , srv1088451917:86 , srv1193919042:86 , srv1508278538:86 , srv1622234:86 , srv1893735558:86 , srv379473650:86 , srv633484942:86 , srv1192186039:630 }
2023-11-19 05:55:18,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:18,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1010680475=1, srv2112263648=52, srv1088451917=4, srv2013791664=48, srv912753136=92, srv1739100857=39, srv363719516=62, srv691060902=79, srv511202588=69, srv681060864=75, srv379473650=64, srv1394095056=18, srv1875881637=42, srv701612762=80, srv788940162=82, srv1339285259=17, srv1248887491=15, srv259232564=56, srv826397409=86, srv818651601=84, srv499650706=68, srv1893735558=45, srv1193919042=12, srv825648330=85, srv1531854253=28, srv1006768162=0, srv2040059558=50, srv569706771=71, srv750618927=81, srv1454213918=23, srv1191824607=10, srv844861984=87, srv569932667=72, srv633484942=73, srv1446552449=22, srv1078978856=3, srv1879791268=43, srv1057583115=2, srv947047334=96, srv1149463047=9, srv340196=59, srv43158002=66, srv477898885=67, srv1954464639=47, srv1699102806=36, srv348394566=61, srv881326790=89, srv1519151996=27, srv2134777287=54, srv248059999=55, srv1677135203=35, srv34152148=60, srv1192186039=11, srv282373973=57, srv1489140336=24, srv682203349=77, srv1948185052=46, srv1196590772=13, srv1116510611=6, srv1752854577=40, srv944214994=94, srv1396821334=19, srv96979101=97, srv1251150576=16, srv1499265910=25, srv1538549932=29, srv686098898=78, srv1622234=32, srv1887558160=44, srv2035246134=49, srv1610207569=31, srv899390246=91, srv1441417436=21, srv1868829410=41, srv283946079=58, srv372182474=63, srv883287252=90, srv1427788070=20, srv1738714834=38, srv933305187=93, srv681314057=76, srv1103983883=5, srv1131905923=7, srv2127642011=53, srv1660337771=34, srv983381455=99, srv419315501=65, srv1232716529=14, srv974230228=98, srv871829116=88, srv667177954=74, srv946474033=95, srv1654052347=33, srv113884954=8, srv547443555=70, srv1508278538=26, srv1558235626=30, srv813378783=83, srv1720750751=37, srv2107064987=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:18,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 05:55:18,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 05:55:18,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 05:55:18,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 05:55:18,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 05:55:18,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 05:55:18,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 05:55:18,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 05:55:18,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 05:55:18,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 05:55:18,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 05:55:18,259 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:18,260 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 05:55:18,260 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.699013996095238E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.994073055283275E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.994073055283275E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=4.166666666666667E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 05:55:18,475 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 312 ms to try 5000 different iterations.  Found a solution that moves 21 regions; Going from a computed imbalance of 4.699013996095238E-5 to a new imbalance of 7.220284439188305E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0023333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:18,475 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv633484942:82 , srv1088451917:83 , srv1193919042:83 , srv1446552449:83 , srv1738714834:83 , srv1893735558:83 , srv282373973:83 , srv682203349:83 , srv1006768162:84 , srv1010680475:84 , srv1057583115:84 , srv1078978856:84 , srv1103983883:84 , srv1116510611:84 , srv1131905923:84 , srv113884954:84 , srv1149463047:84 , srv1191824607:84 , srv1196590772:84 , srv1232716529:84 , srv1251150576:84 , srv1339285259:84 , srv1394095056:84 , srv1396821334:84 , srv1427788070:84 , srv1441417436:84 , srv1454213918:84 , srv1489140336:84 , srv1499265910:84 , srv1508278538:84 , srv1519151996:84 , srv1531854253:84 , srv1538549932:84 , srv1558235626:84 , srv1610207569:84 , srv1622234:84 , srv1654052347:84 , srv1660337771:84 , srv1699102806:84 , srv1720750751:84 , srv1739100857:84 , srv1752854577:84 , srv1868829410:84 , srv1875881637:84 , srv1879791268:84 , srv1887558160:84 , srv1948185052:84 , srv1954464639:84 , srv2013791664:84 , srv2035246134:84 , srv2040059558:84 , srv2112263648:84 , srv2127642011:84 , srv2134777287:84 , srv248059999:84 , srv283946079:84 , srv340196:84 , srv34152148:84 , srv348394566:84 , srv363719516:84 , srv372182474:84 , srv379473650:84 , srv419315501:84 , srv43158002:84 , srv477898885:84 , srv499650706:84 , srv511202588:84 , srv569932667:84 , srv667177954:84 , srv681060864:84 , srv681314057:84 , srv686098898:84 , srv691060902:84 , srv701612762:84 , srv750618927:84 , srv788940162:84 , srv813378783:84 , srv818651601:84 , srv825648330:84 , srv826397409:84 , srv844861984:84 , srv871829116:84 , srv881326790:84 , srv883287252:84 , srv899390246:84 , srv912753136:84 , srv933305187:84 , srv944214994:84 , srv947047334:84 , srv96979101:84 , srv974230228:84 , srv1248887491:85 , srv1677135203:85 , srv2107064987:85 , srv259232564:85 , srv547443555:85 , srv569706771:85 , srv946474033:85 , srv983381455:85 , srv1192186039:685 }
2023-11-19 05:55:18,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:18,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1010680475=1, srv2112263648=52, srv1088451917=4, srv2013791664=48, srv912753136=92, srv1739100857=39, srv363719516=62, srv691060902=79, srv511202588=69, srv681060864=75, srv379473650=64, srv1394095056=18, srv1875881637=42, srv701612762=80, srv788940162=82, srv1339285259=17, srv1248887491=15, srv259232564=56, srv826397409=86, srv818651601=84, srv499650706=68, srv1893735558=45, srv1193919042=12, srv825648330=85, srv1531854253=28, srv1006768162=0, srv2040059558=50, srv569706771=71, srv750618927=81, srv1454213918=23, srv1191824607=10, srv844861984=87, srv569932667=72, srv633484942=73, srv1446552449=22, srv1078978856=3, srv1879791268=43, srv1057583115=2, srv947047334=96, srv1149463047=9, srv340196=59, srv43158002=66, srv477898885=67, srv1954464639=47, srv1699102806=36, srv348394566=61, srv881326790=89, srv1519151996=27, srv2134777287=54, srv248059999=55, srv1677135203=35, srv34152148=60, srv1192186039=11, srv282373973=57, srv1489140336=24, srv682203349=77, srv1948185052=46, srv1196590772=13, srv1116510611=6, srv1752854577=40, srv944214994=94, srv1396821334=19, srv96979101=97, srv1251150576=16, srv1499265910=25, srv1538549932=29, srv686098898=78, srv1622234=32, srv1887558160=44, srv2035246134=49, srv1610207569=31, srv899390246=91, srv1441417436=21, srv1868829410=41, srv283946079=58, srv372182474=63, srv883287252=90, srv1427788070=20, srv1738714834=38, srv933305187=93, srv681314057=76, srv1103983883=5, srv1131905923=7, srv2127642011=53, srv1660337771=34, srv983381455=99, srv419315501=65, srv1232716529=14, srv974230228=98, srv871829116=88, srv667177954=74, srv946474033=95, srv1654052347=33, srv113884954=8, srv547443555=70, srv1508278538=26, srv1558235626=30, srv813378783=83, srv1720750751=37, srv2107064987=51} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:18,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:18,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:18,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:18,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:18,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:18,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 05:55:18,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 05:55:18,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 05:55:18,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 05:55:18,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 05:55:18,606 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:18,606 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv633484942:82 , srv1088451917:83 , srv1193919042:83 , srv1446552449:83 , srv1738714834:83 , srv1893735558:83 , srv282373973:83 , srv682203349:83 , srv1006768162:84 , srv1010680475:84 , srv1057583115:84 , srv1078978856:84 , srv1103983883:84 , srv1116510611:84 , srv1131905923:84 , srv113884954:84 , srv1149463047:84 , srv1191824607:84 , srv1196590772:84 , srv1232716529:84 , srv1251150576:84 , srv1339285259:84 , srv1394095056:84 , srv1396821334:84 , srv1427788070:84 , srv1441417436:84 , srv1454213918:84 , srv1489140336:84 , srv1499265910:84 , srv1508278538:84 , srv1519151996:84 , srv1531854253:84 , srv1538549932:84 , srv1558235626:84 , srv1610207569:84 , srv1622234:84 , srv1654052347:84 , srv1660337771:84 , srv1699102806:84 , srv1720750751:84 , srv1739100857:84 , srv1752854577:84 , srv1868829410:84 , srv1875881637:84 , srv1879791268:84 , srv1887558160:84 , srv1948185052:84 , srv1954464639:84 , srv2013791664:84 , srv2035246134:84 , srv2040059558:84 , srv2112263648:84 , srv2127642011:84 , srv2134777287:84 , srv248059999:84 , srv283946079:84 , srv340196:84 , srv34152148:84 , srv348394566:84 , srv363719516:84 , srv372182474:84 , srv379473650:84 , srv419315501:84 , srv43158002:84 , srv477898885:84 , srv499650706:84 , srv511202588:84 , srv569932667:84 , srv667177954:84 , srv681060864:84 , srv681314057:84 , srv686098898:84 , srv691060902:84 , srv701612762:84 , srv750618927:84 , srv788940162:84 , srv813378783:84 , srv818651601:84 , srv825648330:84 , srv826397409:84 , srv844861984:84 , srv871829116:84 , srv881326790:84 , srv883287252:84 , srv899390246:84 , srv912753136:84 , srv933305187:84 , srv944214994:84 , srv947047334:84 , srv96979101:84 , srv974230228:84 , srv1248887491:85 , srv1677135203:85 , srv2107064987:85 , srv259232564:85 , srv547443555:85 , srv569706771:85 , srv946474033:85 , srv983381455:85 , srv1192186039:685 }
2023-11-19 05:55:18,637 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=163 (was 134) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=17896 (was 18438)
2023-11-19 05:55:18,644 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=163, ProcessCount=168, AvailableMemoryMB=17896
2023-11-19 05:55:18,644 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-19 05:55:18,645 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-19 05:55:18,652 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1026824427:84 , srv1126441253:84 , srv1139079548:84 , srv1189242924:84 , srv1212254994:84 , srv1224360563:84 , srv1283485890:84 , srv141824750:84 , srv1560005668:84 , srv1568793284:84 , srv1572789941:84 , srv1700347448:84 , srv1720779598:84 , srv1790391211:84 , srv1954920532:84 , srv1999751685:84 , srv2010903760:84 , srv23102105:84 , srv232573585:84 , srv28822053:84 , srv362483093:84 , srv41175168:84 , srv488234430:84 , srv525717450:84 , srv658956923:84 , srv786624427:84 , srv79399304:84 , srv924262773:84 , srv959200292:84 , srv966045561:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-19 05:55:18,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:55:18,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1283485890=6, srv488234430=22, srv525717450=23, srv1572789941=10, srv1999751685=15, srv1954920532=14, srv658956923=24, srv362483093=20, srv1560005668=8, srv924262773=27, srv1139079548=2, srv1568793284=9, srv1700347448=11, srv1224360563=5, srv786624427=25, srv2010903760=16, srv1126441253=1, srv1026824427=0, srv141824750=7, srv1790391211=13, srv28822053=19, srv966045561=29, srv1212254994=4, srv1720779598=12, srv959200292=28, srv1189242924=3, srv23102105=17, srv79399304=26, srv232573585=18, srv41175168=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:55:18,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:55:18,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:55:18,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:55:18,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:55:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:55:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:55:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:55:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:55:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:55:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-19 05:55:18,682 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1073107562210491, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:55:18,682 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912531175953114, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1073107562210491, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-19 05:57:18,669 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6332460 different iterations.  Found a solution that moves 1852 regions; Going from a computed imbalance of 0.9912531175953114 to a new imbalance of 4.322660390073086E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6859259259259259, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:57:18,677 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1026824427:90 , srv1126441253:90 , srv1139079548:90 , srv1189242924:90 , srv1212254994:90 , srv1224360563:90 , srv1283485890:90 , srv141824750:90 , srv1560005668:90 , srv1568793284:90 , srv1572789941:90 , srv1700347448:90 , srv1720779598:90 , srv1790391211:90 , srv1954920532:90 , srv1999751685:90 , srv2010903760:90 , srv23102105:90 , srv232573585:90 , srv28822053:90 , srv362483093:90 , srv41175168:90 , srv488234430:90 , srv525717450:90 , srv658956923:90 , srv786624427:90 , srv79399304:90 , srv924262773:90 , srv959200292:90 , srv966045561:90 }
2023-11-19 05:57:18,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 05:57:18,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1283485890=6, srv488234430=22, srv525717450=23, srv1572789941=10, srv1999751685=15, srv1954920532=14, srv658956923=24, srv362483093=20, srv1560005668=8, srv924262773=27, srv1139079548=2, srv1568793284=9, srv1700347448=11, srv1224360563=5, srv786624427=25, srv2010903760=16, srv1126441253=1, srv1026824427=0, srv141824750=7, srv1790391211=13, srv28822053=19, srv966045561=29, srv1212254994=4, srv1720779598=12, srv959200292=28, srv1189242924=3, srv23102105=17, srv79399304=26, srv232573585=18, srv41175168=21} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 05:57:18,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 05:57:18,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-19 05:57:18,696 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 05:57:18,696 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1026824427:90 , srv1126441253:90 , srv1139079548:90 , srv1189242924:90 , srv1212254994:90 , srv1224360563:90 , srv1283485890:90 , srv141824750:90 , srv1560005668:90 , srv1568793284:90 , srv1572789941:90 , srv1700347448:90 , srv1720779598:90 , srv1790391211:90 , srv1954920532:90 , srv1999751685:90 , srv2010903760:90 , srv23102105:90 , srv232573585:90 , srv28822053:90 , srv362483093:90 , srv41175168:90 , srv488234430:90 , srv525717450:90 , srv658956923:90 , srv786624427:90 , srv79399304:90 , srv924262773:90 , srv959200292:90 , srv966045561:90 }
2023-11-19 05:57:18,709 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=210 (was 163) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=17957 (was 17896) - AvailableMemoryMB LEAK? -