2023-04-09 07:56:40,442 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-09 07:56:40,774 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-09 07:56:40,901 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-09 07:56:40,969 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-09 07:56:40,976 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-09 07:56:40,988 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=382, ProcessCount=169, AvailableMemoryMB=2571
2023-04-09 07:56:40,990 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-09 07:56:40,990 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-09 07:56:41,102 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1021294430:84 , srv1059843103:84 , srv1185238484:84 , srv1203558936:84 , srv1226351106:84 , srv1242568714:84 , srv1249900130:84 , srv1259880440:84 , srv1272107824:84 , srv1302297482:84 , srv1324187211:84 , srv1349863828:84 , srv1374458467:84 , srv1404140323:84 , srv1416059560:84 , srv1436883760:84 , srv14386197:84 , srv1455099762:84 , srv147035053:84 , srv1484190641:84 , srv1496323019:84 , srv153326775:84 , srv1553693449:84 , srv1571849647:84 , srv1588012425:84 , srv160804212:84 , srv1613111278:84 , srv1620917237:84 , srv1640248240:84 , srv1671000402:84 , srv1706800158:84 , srv171846935:84 , srv1757331731:84 , srv1758538391:84 , srv1806728745:84 , srv1817523003:84 , srv1837087790:84 , srv1882148262:84 , srv1885287964:84 , srv1937406611:84 , srv1959453502:84 , srv1959898569:84 , srv1963790028:84 , srv1974164516:84 , srv1984167972:84 , srv1993176885:84 , srv1998156063:84 , srv2024631389:84 , srv2024646297:84 , srv2112672984:84 , srv2114623986:84 , srv213249776:84 , srv214077275:84 , srv2144106903:84 , srv223845191:84 , srv225308941:84 , srv237060770:84 , srv284868531:84 , srv303083600:84 , srv343565501:84 , srv346071689:84 , srv362306709:84 , srv365801085:84 , srv36656664:84 , srv37604828:84 , srv429937992:84 , srv444350700:84 , srv45536702:84 , srv469551865:84 , srv486403198:84 , srv498532092:84 , srv499062401:84 , srv500164702:84 , srv504457475:84 , srv512172046:84 , srv518130791:84 , srv543370116:84 , srv548894497:84 , srv554399912:84 , srv558409142:84 , srv575683987:84 , srv58651995:84 , srv627636797:84 , srv628181591:84 , srv634740791:84 , srv64022472:84 , srv658906044:84 , srv689712431:84 , srv745964880:84 , srv746433106:84 , srv762896858:84 , srv770107721:84 , srv877712440:84 , srv933753265:84 , srv935368267:84 , srv943428085:84 , srv945389267:84 , srv955938013:84 , srv98582732:84 , srv736875087:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-09 07:56:41,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:41,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1640248240=28, srv1817523003=35, srv1974164516=43, srv1706800158=30, srv1588012425=24, srv1837087790=36, srv745964880=89, srv1259880440=7, srv1806728745=34, srv1553693449=22, srv1436883760=15, srv1455099762=17, srv1249900130=6, srv634740791=84, srv343565501=59, srv575683987=80, srv518130791=75, srv1021294430=0, srv746433106=90, srv1496323019=20, srv1993176885=45, srv1349863828=11, srv2144106903=53, srv500164702=72, srv762896858=91, srv658906044=86, srv945389267=97, srv943428085=96, srv689712431=87, srv98582732=99, srv36656664=63, srv1226351106=4, srv1671000402=29, srv498532092=70, srv2024631389=47, srv548894497=77, srv2114623986=50, srv171846935=31, srv1984167972=44, srv1959898569=41, srv1324187211=10, srv1404140323=13, srv213249776=51, srv362306709=61, srv45536702=67, srv1937406611=39, srv429937992=65, srv554399912=78, srv223845191=54, srv365801085=62, srv1059843103=1, srv225308941=55, srv935368267=95, srv237060770=56, srv153326775=21, srv1203558936=3, srv1571849647=23, srv1882148262=37, srv512172046=74, srv1998156063=46, srv1963790028=42, srv486403198=69, srv1959453502=40, srv2024646297=48, srv1620917237=27, srv444350700=66, srv2112672984=49, srv558409142=79, srv1484190641=19, srv284868531=57, srv214077275=52, srv499062401=71, srv58651995=81, srv147035053=18, srv1757331731=32, srv303083600=58, srv628181591=83, srv955938013=98, srv1758538391=33, srv1374458467=12, srv543370116=76, srv736875087=88, srv64022472=85, srv14386197=16, srv1885287964=38, srv1242568714=5, srv1613111278=26, srv160804212=25, srv877712440=93, srv1302297482=9, srv933753265=94, srv627636797=82, srv770107721=92, srv37604828=64, srv1416059560=14, srv1272107824=8, srv504457475=73, srv1185238484=2, srv469551865=68, srv346071689=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:41,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:41,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:41,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-09 07:56:41,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-09 07:56:41,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-09 07:56:41,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-09 07:56:41,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-09 07:56:41,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-09 07:56:41,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-09 07:56:41,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-09 07:56:41,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-09 07:56:41,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-09 07:56:41,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-09 07:56:41,379 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:41,379 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-09 07:56:41,380 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-09 07:56:41,693 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 458 ms to try 5000 different iterations.  Found a solution that moves 4703 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17907965720508762. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04245608339606742, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0588517130608005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5225555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04245608339606742, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17475, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23608333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:41,720 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1758538391:81 , srv1959898569:81 , srv548894497:81 , srv58651995:81 , srv943428085:81 , srv1998156063:82 , srv223845191:82 , srv486403198:82 , srv64022472:82 , srv1203558936:83 , srv1259880440:83 , srv1620917237:83 , srv1885287964:83 , srv343565501:83 , srv36656664:83 , srv500164702:83 , srv745964880:83 , srv762896858:83 , srv1059843103:84 , srv1249900130:84 , srv1404140323:84 , srv1436883760:84 , srv1571849647:84 , srv1817523003:84 , srv1963790028:84 , srv2114623986:84 , srv213249776:84 , srv429937992:84 , srv498532092:84 , srv512172046:84 , srv518130791:84 , srv554399912:84 , srv658906044:84 , srv746433106:84 , srv1242568714:85 , srv1324187211:85 , srv153326775:85 , srv1937406611:85 , srv1974164516:85 , srv2024646297:85 , srv362306709:85 , srv365801085:85 , srv444350700:85 , srv45536702:85 , srv1302297482:86 , srv1455099762:86 , srv1553693449:86 , srv1671000402:86 , srv1757331731:86 , srv1837087790:86 , srv1882148262:86 , srv1993176885:86 , srv37604828:86 , srv499062401:86 , srv575683987:86 , srv770107721:86 , srv877712440:86 , srv945389267:86 , srv98582732:86 , srv1349863828:87 , srv1416059560:87 , srv1588012425:87 , srv171846935:87 , srv2112672984:87 , srv284868531:87 , srv628181591:87 , srv933753265:87 , srv955938013:87 , srv1272107824:88 , srv1374458467:88 , srv160804212:88 , srv1706800158:88 , srv1984167972:88 , srv237060770:88 , srv504457475:88 , srv558409142:88 , srv627636797:88 , srv689712431:88 , srv14386197:89 , srv147035053:89 , srv1484190641:89 , srv1959453502:89 , srv2024631389:89 , srv346071689:89 , srv634740791:89 , srv1021294430:90 , srv1496323019:90 , srv1613111278:90 , srv1640248240:90 , srv543370116:90 , srv1185238484:91 , srv214077275:91 , srv2144106903:91 , srv303083600:91 , srv935368267:91 , srv1226351106:92 , srv1806728745:92 , srv225308941:95 , srv469551865:99 , srv736875087:467 }
2023-04-09 07:56:41,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:41,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1640248240=28, srv1817523003=35, srv1974164516=43, srv1706800158=30, srv1588012425=24, srv1837087790=36, srv745964880=89, srv1259880440=7, srv1806728745=34, srv1553693449=22, srv1436883760=15, srv1455099762=17, srv1249900130=6, srv634740791=84, srv343565501=59, srv575683987=80, srv518130791=75, srv1021294430=0, srv746433106=90, srv1496323019=20, srv1993176885=45, srv1349863828=11, srv2144106903=53, srv500164702=72, srv762896858=91, srv658906044=86, srv945389267=97, srv943428085=96, srv689712431=87, srv98582732=99, srv36656664=63, srv1226351106=4, srv1671000402=29, srv498532092=70, srv2024631389=47, srv548894497=77, srv2114623986=50, srv171846935=31, srv1984167972=44, srv1959898569=41, srv1324187211=10, srv1404140323=13, srv213249776=51, srv362306709=61, srv45536702=67, srv1937406611=39, srv429937992=65, srv554399912=78, srv223845191=54, srv365801085=62, srv1059843103=1, srv225308941=55, srv935368267=95, srv237060770=56, srv153326775=21, srv1203558936=3, srv1571849647=23, srv1882148262=37, srv512172046=74, srv1998156063=46, srv1963790028=42, srv486403198=69, srv1959453502=40, srv2024646297=48, srv1620917237=27, srv444350700=66, srv2112672984=49, srv558409142=79, srv1484190641=19, srv284868531=57, srv214077275=52, srv499062401=71, srv58651995=81, srv147035053=18, srv1757331731=32, srv303083600=58, srv628181591=83, srv955938013=98, srv1758538391=33, srv1374458467=12, srv543370116=76, srv736875087=88, srv64022472=85, srv14386197=16, srv1885287964=38, srv1242568714=5, srv1613111278=26, srv160804212=25, srv877712440=93, srv1302297482=9, srv933753265=94, srv627636797=82, srv770107721=92, srv37604828=64, srv1416059560=14, srv1272107824=8, srv504457475=73, srv1185238484=2, srv469551865=68, srv346071689=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:41,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:41,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-09 07:56:41,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-09 07:56:41,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-09 07:56:41,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-09 07:56:41,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-09 07:56:41,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-09 07:56:41,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-09 07:56:41,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-09 07:56:41,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-09 07:56:41,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-09 07:56:41,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-09 07:56:41,873 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04245608339606742, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0588517130608005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04245608339606742, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17475, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23608333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:41,873 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-09 07:56:41,873 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17904672609523692, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04245608339606742, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0588517130608005, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04245608339606742, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17475, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23608333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-09 07:56:42,144 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 380 ms to try 5000 different iterations.  Found a solution that moves 2632 regions; Going from a computed imbalance of 0.17904672609523692 to a new imbalance of 0.0169680778323483. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294666522760908); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03587920875174024, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29244444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294666522760908); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:42,155 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv469551865:72 , srv225308941:75 , srv1226351106:80 , srv1021294430:81 , srv1640248240:81 , srv1959453502:81 , srv543370116:81 , srv935368267:81 , srv1185238484:82 , srv147035053:82 , srv1806728745:82 , srv346071689:82 , srv558409142:82 , srv634740791:82 , srv1302297482:83 , srv1484190641:83 , srv1496323019:83 , srv1588012425:83 , srv160804212:83 , srv1984167972:83 , srv2144106903:83 , srv504457475:83 , srv627636797:83 , srv689712431:83 , srv933753265:83 , srv1374458467:84 , srv1416059560:84 , srv1613111278:84 , srv1706800158:84 , srv2024631389:84 , srv214077275:84 , srv237060770:84 , srv303083600:84 , srv945389267:84 , srv1349863828:85 , srv14386197:85 , srv1837087790:85 , srv2024646297:85 , srv213249776:85 , srv284868531:85 , srv37604828:85 , srv575683987:85 , srv628181591:85 , srv955938013:85 , srv1242568714:86 , srv1272107824:86 , srv1324187211:86 , srv1455099762:86 , srv153326775:86 , srv171846935:86 , srv658906044:86 , srv98582732:86 , srv1059843103:87 , srv1404140323:87 , srv1553693449:87 , srv1671000402:87 , srv1882148262:87 , srv2112672984:87 , srv2114623986:87 , srv362306709:87 , srv365801085:87 , srv36656664:87 , srv45536702:87 , srv499062401:87 , srv877712440:87 , srv1259880440:88 , srv1436883760:88 , srv1571849647:88 , srv1757331731:88 , srv1817523003:88 , srv1937406611:88 , srv1974164516:88 , srv223845191:88 , srv343565501:88 , srv429937992:88 , srv498532092:88 , srv512172046:88 , srv518130791:88 , srv554399912:88 , srv58651995:88 , srv745964880:88 , srv770107721:88 , srv1249900130:89 , srv1620917237:89 , srv1993176885:89 , srv1998156063:89 , srv444350700:89 , srv746433106:89 , srv1203558936:90 , srv1959898569:90 , srv500164702:90 , srv64022472:90 , srv762896858:90 , srv1758538391:91 , srv1885287964:91 , srv1963790028:91 , srv486403198:91 , srv548894497:91 , srv943428085:92 , srv736875087:511 }
2023-04-09 07:56:42,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:42,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1640248240=28, srv1817523003=35, srv1974164516=43, srv1706800158=30, srv1588012425=24, srv1837087790=36, srv745964880=89, srv1259880440=7, srv1806728745=34, srv1553693449=22, srv1436883760=15, srv1455099762=17, srv1249900130=6, srv634740791=84, srv343565501=59, srv575683987=80, srv518130791=75, srv1021294430=0, srv746433106=90, srv1496323019=20, srv1993176885=45, srv1349863828=11, srv2144106903=53, srv500164702=72, srv762896858=91, srv658906044=86, srv945389267=97, srv943428085=96, srv689712431=87, srv98582732=99, srv36656664=63, srv1226351106=4, srv1671000402=29, srv498532092=70, srv2024631389=47, srv548894497=77, srv2114623986=50, srv171846935=31, srv1984167972=44, srv1959898569=41, srv1324187211=10, srv1404140323=13, srv213249776=51, srv362306709=61, srv45536702=67, srv1937406611=39, srv429937992=65, srv554399912=78, srv223845191=54, srv365801085=62, srv1059843103=1, srv225308941=55, srv935368267=95, srv237060770=56, srv153326775=21, srv1203558936=3, srv1571849647=23, srv1882148262=37, srv512172046=74, srv1998156063=46, srv1963790028=42, srv486403198=69, srv1959453502=40, srv2024646297=48, srv1620917237=27, srv444350700=66, srv2112672984=49, srv558409142=79, srv1484190641=19, srv284868531=57, srv214077275=52, srv499062401=71, srv58651995=81, srv147035053=18, srv1757331731=32, srv303083600=58, srv628181591=83, srv955938013=98, srv1758538391=33, srv1374458467=12, srv543370116=76, srv736875087=88, srv64022472=85, srv14386197=16, srv1885287964=38, srv1242568714=5, srv1613111278=26, srv160804212=25, srv877712440=93, srv1302297482=9, srv933753265=94, srv627636797=82, srv770107721=92, srv37604828=64, srv1416059560=14, srv1272107824=8, srv504457475=73, srv1185238484=2, srv469551865=68, srv346071689=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:42,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:42,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-09 07:56:42,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-09 07:56:42,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-09 07:56:42,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-09 07:56:42,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-09 07:56:42,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-09 07:56:42,305 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294666522760908); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03587920875174024, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294666522760908); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:42,306 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-09 07:56:42,306 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016949648174443322, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02294666522760908); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03587920875174024, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02294666522760908); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04525, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-09 07:56:42,552 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 350 ms to try 5000 different iterations.  Found a solution that moves 1044 regions; Going from a computed imbalance of 0.016949648174443322 to a new imbalance of 7.495773516120664E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007672040602626861); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008355190191882916); MoveCostFunction : (multiplier=7.0, imbalance=0.116, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007672040602626861); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:42,554 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1806728745:82 , srv1885287964:82 , srv1963790028:82 , srv303083600:82 , srv1272107824:83 , srv14386197:83 , srv1993176885:83 , srv214077275:83 , srv2144106903:83 , srv1249900130:84 , srv1496323019:84 , srv1553693449:84 , srv1613111278:84 , srv1671000402:84 , srv1757331731:84 , srv1882148262:84 , srv1937406611:84 , srv1974164516:84 , srv2112672984:84 , srv444350700:84 , srv486403198:84 , srv500164702:84 , srv628181591:84 , srv746433106:84 , srv943428085:84 , srv955938013:84 , srv1185238484:85 , srv1203558936:85 , srv1226351106:85 , srv1349863828:85 , srv1404140323:85 , srv1455099762:85 , srv1484190641:85 , srv1571849647:85 , srv1620917237:85 , srv171846935:85 , srv1758538391:85 , srv1837087790:85 , srv1984167972:85 , srv284868531:85 , srv365801085:85 , srv429937992:85 , srv498532092:85 , srv499062401:85 , srv512172046:85 , srv518130791:85 , srv575683987:85 , srv770107721:85 , srv877712440:85 , srv935368267:85 , srv98582732:85 , srv1021294430:86 , srv1242568714:86 , srv1259880440:86 , srv1324187211:86 , srv1374458467:86 , srv1416059560:86 , srv1436883760:86 , srv1706800158:86 , srv1817523003:86 , srv1959453502:86 , srv1998156063:86 , srv2024631389:86 , srv2114623986:86 , srv237060770:86 , srv343565501:86 , srv346071689:86 , srv362306709:86 , srv469551865:86 , srv504457475:86 , srv543370116:86 , srv548894497:86 , srv554399912:86 , srv627636797:86 , srv634740791:86 , srv64022472:86 , srv745964880:86 , srv762896858:86 , srv1059843103:87 , srv147035053:87 , srv153326775:87 , srv1588012425:87 , srv160804212:87 , srv1640248240:87 , srv1959898569:87 , srv2024646297:87 , srv225308941:87 , srv36656664:87 , srv37604828:87 , srv45536702:87 , srv58651995:87 , srv689712431:87 , srv945389267:87 , srv1302297482:88 , srv223845191:88 , srv558409142:88 , srv658906044:88 , srv933753265:88 , srv213249776:89 , srv736875087:548 }
2023-04-09 07:56:42,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:42,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1640248240=28, srv1817523003=35, srv1974164516=43, srv1706800158=30, srv1588012425=24, srv1837087790=36, srv745964880=89, srv1259880440=7, srv1806728745=34, srv1553693449=22, srv1436883760=15, srv1455099762=17, srv1249900130=6, srv634740791=84, srv343565501=59, srv575683987=80, srv518130791=75, srv1021294430=0, srv746433106=90, srv1496323019=20, srv1993176885=45, srv1349863828=11, srv2144106903=53, srv500164702=72, srv762896858=91, srv658906044=86, srv945389267=97, srv943428085=96, srv689712431=87, srv98582732=99, srv36656664=63, srv1226351106=4, srv1671000402=29, srv498532092=70, srv2024631389=47, srv548894497=77, srv2114623986=50, srv171846935=31, srv1984167972=44, srv1959898569=41, srv1324187211=10, srv1404140323=13, srv213249776=51, srv362306709=61, srv45536702=67, srv1937406611=39, srv429937992=65, srv554399912=78, srv223845191=54, srv365801085=62, srv1059843103=1, srv225308941=55, srv935368267=95, srv237060770=56, srv153326775=21, srv1203558936=3, srv1571849647=23, srv1882148262=37, srv512172046=74, srv1998156063=46, srv1963790028=42, srv486403198=69, srv1959453502=40, srv2024646297=48, srv1620917237=27, srv444350700=66, srv2112672984=49, srv558409142=79, srv1484190641=19, srv284868531=57, srv214077275=52, srv499062401=71, srv58651995=81, srv147035053=18, srv1757331731=32, srv303083600=58, srv628181591=83, srv955938013=98, srv1758538391=33, srv1374458467=12, srv543370116=76, srv736875087=88, srv64022472=85, srv14386197=16, srv1885287964=38, srv1242568714=5, srv1613111278=26, srv160804212=25, srv877712440=93, srv1302297482=9, srv933753265=94, srv627636797=82, srv770107721=92, srv37604828=64, srv1416059560=14, srv1272107824=8, srv504457475=73, srv1185238484=2, srv469551865=68, srv346071689=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:42,571 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:42,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-09 07:56:42,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-09 07:56:42,574 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-09 07:56:42,575 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-09 07:56:42,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-09 07:56:42,577 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-09 07:56:42,688 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007672040602626861); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008355190191882916); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007672040602626861); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:42,688 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-09 07:56:42,688 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.42267107367083E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007672040602626861); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008355190191882916); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007672040602626861); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-09 07:56:43,032 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 444 ms to try 5000 different iterations.  Found a solution that moves 311 regions; Going from a computed imbalance of 7.42267107367083E-4 to a new imbalance of 3.623462125469782E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.034555555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:43,033 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv770107721:82 , srv1374458467:83 , srv45536702:83 , srv1059843103:84 , srv1185238484:84 , srv1203558936:84 , srv1436883760:84 , srv147035053:84 , srv1588012425:84 , srv1613111278:84 , srv1640248240:84 , srv1706800158:84 , srv1757331731:84 , srv1817523003:84 , srv1882148262:84 , srv1993176885:84 , srv2024631389:84 , srv2024646297:84 , srv2112672984:84 , srv213249776:84 , srv214077275:84 , srv223845191:84 , srv237060770:84 , srv362306709:84 , srv37604828:84 , srv444350700:84 , srv499062401:84 , srv548894497:84 , srv554399912:84 , srv558409142:84 , srv64022472:84 , srv658906044:84 , srv689712431:84 , srv762896858:84 , srv877712440:84 , srv933753265:84 , srv98582732:84 , srv1021294430:85 , srv1226351106:85 , srv1242568714:85 , srv1249900130:85 , srv1259880440:85 , srv1272107824:85 , srv1302297482:85 , srv1324187211:85 , srv1349863828:85 , srv1416059560:85 , srv14386197:85 , srv1455099762:85 , srv1484190641:85 , srv1496323019:85 , srv153326775:85 , srv1553693449:85 , srv1571849647:85 , srv160804212:85 , srv1620917237:85 , srv1671000402:85 , srv171846935:85 , srv1758538391:85 , srv1959453502:85 , srv1959898569:85 , srv1963790028:85 , srv1974164516:85 , srv1998156063:85 , srv2114623986:85 , srv2144106903:85 , srv225308941:85 , srv284868531:85 , srv303083600:85 , srv343565501:85 , srv346071689:85 , srv365801085:85 , srv36656664:85 , srv429937992:85 , srv469551865:85 , srv486403198:85 , srv498532092:85 , srv500164702:85 , srv504457475:85 , srv512172046:85 , srv518130791:85 , srv543370116:85 , srv627636797:85 , srv634740791:85 , srv745964880:85 , srv746433106:85 , srv935368267:85 , srv943428085:85 , srv945389267:85 , srv1404140323:86 , srv1806728745:86 , srv1837087790:86 , srv1885287964:86 , srv1937406611:86 , srv1984167972:86 , srv575683987:86 , srv58651995:86 , srv628181591:86 , srv955938013:86 , srv736875087:616 }
2023-04-09 07:56:43,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:43,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1640248240=28, srv1817523003=35, srv1974164516=43, srv1706800158=30, srv1588012425=24, srv1837087790=36, srv745964880=89, srv1259880440=7, srv1806728745=34, srv1553693449=22, srv1436883760=15, srv1455099762=17, srv1249900130=6, srv634740791=84, srv343565501=59, srv575683987=80, srv518130791=75, srv1021294430=0, srv746433106=90, srv1496323019=20, srv1993176885=45, srv1349863828=11, srv2144106903=53, srv500164702=72, srv762896858=91, srv658906044=86, srv945389267=97, srv943428085=96, srv689712431=87, srv98582732=99, srv36656664=63, srv1226351106=4, srv1671000402=29, srv498532092=70, srv2024631389=47, srv548894497=77, srv2114623986=50, srv171846935=31, srv1984167972=44, srv1959898569=41, srv1324187211=10, srv1404140323=13, srv213249776=51, srv362306709=61, srv45536702=67, srv1937406611=39, srv429937992=65, srv554399912=78, srv223845191=54, srv365801085=62, srv1059843103=1, srv225308941=55, srv935368267=95, srv237060770=56, srv153326775=21, srv1203558936=3, srv1571849647=23, srv1882148262=37, srv512172046=74, srv1998156063=46, srv1963790028=42, srv486403198=69, srv1959453502=40, srv2024646297=48, srv1620917237=27, srv444350700=66, srv2112672984=49, srv558409142=79, srv1484190641=19, srv284868531=57, srv214077275=52, srv499062401=71, srv58651995=81, srv147035053=18, srv1757331731=32, srv303083600=58, srv628181591=83, srv955938013=98, srv1758538391=33, srv1374458467=12, srv543370116=76, srv736875087=88, srv64022472=85, srv14386197=16, srv1885287964=38, srv1242568714=5, srv1613111278=26, srv160804212=25, srv877712440=93, srv1302297482=9, srv933753265=94, srv627636797=82, srv770107721=92, srv37604828=64, srv1416059560=14, srv1272107824=8, srv504457475=73, srv1185238484=2, srv469551865=68, srv346071689=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:43,049 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:43,050 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-09 07:56:43,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-09 07:56:43,052 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-09 07:56:43,053 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-09 07:56:43,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-09 07:56:43,155 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:43,155 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-09 07:56:43,156 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.405695271045473E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-09 07:56:43,374 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 311 ms to try 5000 different iterations.  Found a solution that moves 36 regions; Going from a computed imbalance of 3.405695271045473E-5 to a new imbalance of 6.650063060027741E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.004); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:43,375 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv153326775:83 , srv160804212:83 , srv171846935:83 , srv1937406611:83 , srv1959898569:83 , srv2024631389:83 , srv762896858:83 , srv1021294430:84 , srv1059843103:84 , srv1185238484:84 , srv1203558936:84 , srv1226351106:84 , srv1242568714:84 , srv1249900130:84 , srv1259880440:84 , srv1272107824:84 , srv1302297482:84 , srv1324187211:84 , srv1349863828:84 , srv1404140323:84 , srv1416059560:84 , srv1436883760:84 , srv14386197:84 , srv1455099762:84 , srv147035053:84 , srv1484190641:84 , srv1496323019:84 , srv1553693449:84 , srv1571849647:84 , srv1613111278:84 , srv1620917237:84 , srv1640248240:84 , srv1671000402:84 , srv1706800158:84 , srv1757331731:84 , srv1758538391:84 , srv1806728745:84 , srv1817523003:84 , srv1837087790:84 , srv1885287964:84 , srv1963790028:84 , srv1974164516:84 , srv1984167972:84 , srv1993176885:84 , srv1998156063:84 , srv2112672984:84 , srv2114623986:84 , srv213249776:84 , srv214077275:84 , srv2144106903:84 , srv223845191:84 , srv225308941:84 , srv237060770:84 , srv284868531:84 , srv303083600:84 , srv343565501:84 , srv346071689:84 , srv362306709:84 , srv365801085:84 , srv36656664:84 , srv37604828:84 , srv429937992:84 , srv444350700:84 , srv45536702:84 , srv469551865:84 , srv486403198:84 , srv498532092:84 , srv499062401:84 , srv500164702:84 , srv504457475:84 , srv512172046:84 , srv518130791:84 , srv543370116:84 , srv548894497:84 , srv554399912:84 , srv558409142:84 , srv575683987:84 , srv58651995:84 , srv627636797:84 , srv628181591:84 , srv634740791:84 , srv64022472:84 , srv658906044:84 , srv689712431:84 , srv745964880:84 , srv746433106:84 , srv877712440:84 , srv933753265:84 , srv935368267:84 , srv943428085:84 , srv945389267:84 , srv955938013:84 , srv1374458467:85 , srv1588012425:85 , srv1882148262:85 , srv1959453502:85 , srv2024646297:85 , srv770107721:85 , srv98582732:85 , srv736875087:684 }
2023-04-09 07:56:43,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:43,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1640248240=28, srv1817523003=35, srv1974164516=43, srv1706800158=30, srv1588012425=24, srv1837087790=36, srv745964880=89, srv1259880440=7, srv1806728745=34, srv1553693449=22, srv1436883760=15, srv1455099762=17, srv1249900130=6, srv634740791=84, srv343565501=59, srv575683987=80, srv518130791=75, srv1021294430=0, srv746433106=90, srv1496323019=20, srv1993176885=45, srv1349863828=11, srv2144106903=53, srv500164702=72, srv762896858=91, srv658906044=86, srv945389267=97, srv943428085=96, srv689712431=87, srv98582732=99, srv36656664=63, srv1226351106=4, srv1671000402=29, srv498532092=70, srv2024631389=47, srv548894497=77, srv2114623986=50, srv171846935=31, srv1984167972=44, srv1959898569=41, srv1324187211=10, srv1404140323=13, srv213249776=51, srv362306709=61, srv45536702=67, srv1937406611=39, srv429937992=65, srv554399912=78, srv223845191=54, srv365801085=62, srv1059843103=1, srv225308941=55, srv935368267=95, srv237060770=56, srv153326775=21, srv1203558936=3, srv1571849647=23, srv1882148262=37, srv512172046=74, srv1998156063=46, srv1963790028=42, srv486403198=69, srv1959453502=40, srv2024646297=48, srv1620917237=27, srv444350700=66, srv2112672984=49, srv558409142=79, srv1484190641=19, srv284868531=57, srv214077275=52, srv499062401=71, srv58651995=81, srv147035053=18, srv1757331731=32, srv303083600=58, srv628181591=83, srv955938013=98, srv1758538391=33, srv1374458467=12, srv543370116=76, srv736875087=88, srv64022472=85, srv14386197=16, srv1885287964=38, srv1242568714=5, srv1613111278=26, srv160804212=25, srv877712440=93, srv1302297482=9, srv933753265=94, srv627636797=82, srv770107721=92, srv37604828=64, srv1416059560=14, srv1272107824=8, srv504457475=73, srv1185238484=2, srv469551865=68, srv346071689=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:43,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:43,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:43,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:43,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:43,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:43,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:43,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-09 07:56:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-09 07:56:43,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-09 07:56:43,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-09 07:56:43,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-09 07:56:43,497 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:43,498 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv153326775:83 , srv160804212:83 , srv171846935:83 , srv1937406611:83 , srv1959898569:83 , srv2024631389:83 , srv762896858:83 , srv1021294430:84 , srv1059843103:84 , srv1185238484:84 , srv1203558936:84 , srv1226351106:84 , srv1242568714:84 , srv1249900130:84 , srv1259880440:84 , srv1272107824:84 , srv1302297482:84 , srv1324187211:84 , srv1349863828:84 , srv1404140323:84 , srv1416059560:84 , srv1436883760:84 , srv14386197:84 , srv1455099762:84 , srv147035053:84 , srv1484190641:84 , srv1496323019:84 , srv1553693449:84 , srv1571849647:84 , srv1613111278:84 , srv1620917237:84 , srv1640248240:84 , srv1671000402:84 , srv1706800158:84 , srv1757331731:84 , srv1758538391:84 , srv1806728745:84 , srv1817523003:84 , srv1837087790:84 , srv1885287964:84 , srv1963790028:84 , srv1974164516:84 , srv1984167972:84 , srv1993176885:84 , srv1998156063:84 , srv2112672984:84 , srv2114623986:84 , srv213249776:84 , srv214077275:84 , srv2144106903:84 , srv223845191:84 , srv225308941:84 , srv237060770:84 , srv284868531:84 , srv303083600:84 , srv343565501:84 , srv346071689:84 , srv362306709:84 , srv365801085:84 , srv36656664:84 , srv37604828:84 , srv429937992:84 , srv444350700:84 , srv45536702:84 , srv469551865:84 , srv486403198:84 , srv498532092:84 , srv499062401:84 , srv500164702:84 , srv504457475:84 , srv512172046:84 , srv518130791:84 , srv543370116:84 , srv548894497:84 , srv554399912:84 , srv558409142:84 , srv575683987:84 , srv58651995:84 , srv627636797:84 , srv628181591:84 , srv634740791:84 , srv64022472:84 , srv658906044:84 , srv689712431:84 , srv745964880:84 , srv746433106:84 , srv877712440:84 , srv933753265:84 , srv935368267:84 , srv943428085:84 , srv945389267:84 , srv955938013:84 , srv1374458467:85 , srv1588012425:85 , srv1882148262:85 , srv1959453502:85 , srv2024646297:85 , srv770107721:85 , srv98582732:85 , srv736875087:684 }
2023-04-09 07:56:43,524 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=382 (was 382), ProcessCount=169 (was 169), AvailableMemoryMB=2071 (was 2571)
2023-04-09 07:56:43,533 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=382, ProcessCount=169, AvailableMemoryMB=2071
2023-04-09 07:56:43,534 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-09 07:56:43,534 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-09 07:56:43,541 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035910089:84 , srv108793106:84 , srv1126286933:84 , srv1232136:84 , srv1325105585:84 , srv1447785932:84 , srv1591726071:84 , srv1610647168:84 , srv1660715895:84 , srv1666118195:84 , srv1668707718:84 , srv1684078945:84 , srv187626031:84 , srv1878801016:84 , srv1906876835:84 , srv2006078882:84 , srv442006915:84 , srv46496116:84 , srv517181553:84 , srv584177592:84 , srv595332700:84 , srv685791895:84 , srv715192276:84 , srv720678095:84 , srv835782153:84 , srv89703509:84 , srv964688791:84 , srv985506689:84 , srv994401893:84 , srv438438309:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-09 07:56:43,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:56:43,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv442006915=17, srv715192276=23, srv835782153=25, srv438438309=16, srv1035910089=0, srv1325105585=4, srv1878801016=13, srv1666118195=9, srv1684078945=11, srv1126286933=2, srv1447785932=5, srv1610647168=7, srv584177592=20, srv2006078882=15, srv1660715895=8, srv46496116=18, srv994401893=29, srv1591726071=6, srv517181553=19, srv108793106=1, srv1668707718=10, srv89703509=26, srv720678095=24, srv964688791=27, srv985506689=28, srv1906876835=14, srv685791895=22, srv1232136=3, srv595332700=21, srv187626031=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:56:43,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:56:43,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:56:43,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-09 07:56:43,567 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.059299229345335, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:56:43,567 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912379893199651, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.059299229345335, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-09 07:58:43,555 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6063236 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912379893199651 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:58:43,563 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035910089:90 , srv108793106:90 , srv1126286933:90 , srv1232136:90 , srv1325105585:90 , srv1447785932:90 , srv1591726071:90 , srv1610647168:90 , srv1660715895:90 , srv1666118195:90 , srv1668707718:90 , srv1684078945:90 , srv187626031:90 , srv1878801016:90 , srv1906876835:90 , srv2006078882:90 , srv438438309:90 , srv442006915:90 , srv46496116:90 , srv517181553:90 , srv584177592:90 , srv595332700:90 , srv685791895:90 , srv715192276:90 , srv720678095:90 , srv835782153:90 , srv89703509:90 , srv964688791:90 , srv985506689:90 , srv994401893:90 }
2023-04-09 07:58:43,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-09 07:58:43,565 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv442006915=17, srv715192276=23, srv835782153=25, srv438438309=16, srv1035910089=0, srv1325105585=4, srv1878801016=13, srv1666118195=9, srv1684078945=11, srv1126286933=2, srv1447785932=5, srv1610647168=7, srv584177592=20, srv2006078882=15, srv1660715895=8, srv46496116=18, srv994401893=29, srv1591726071=6, srv517181553=19, srv108793106=1, srv1668707718=10, srv89703509=26, srv720678095=24, srv964688791=27, srv985506689=28, srv1906876835=14, srv685791895=22, srv1232136=3, srv595332700=21, srv187626031=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-09 07:58:43,569 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-09 07:58:43,570 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-09 07:58:43,587 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-09 07:58:43,587 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035910089:90 , srv108793106:90 , srv1126286933:90 , srv1232136:90 , srv1325105585:90 , srv1447785932:90 , srv1591726071:90 , srv1610647168:90 , srv1660715895:90 , srv1666118195:90 , srv1668707718:90 , srv1684078945:90 , srv187626031:90 , srv1878801016:90 , srv1906876835:90 , srv2006078882:90 , srv438438309:90 , srv442006915:90 , srv46496116:90 , srv517181553:90 , srv584177592:90 , srv595332700:90 , srv685791895:90 , srv715192276:90 , srv720678095:90 , srv835782153:90 , srv89703509:90 , srv964688791:90 , srv985506689:90 , srv994401893:90 }
2023-04-09 07:58:43,602 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 382), ProcessCount=169 (was 169), AvailableMemoryMB=2384 (was 2071) - AvailableMemoryMB LEAK? -