2023-07-12 18:59:22,575 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-12 18:59:22,807 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-12 18:59:22,894 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-12 18:59:22,945 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-12 18:59:22,951 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-12 18:59:22,962 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=144, ProcessCount=164, AvailableMemoryMB=6704
2023-07-12 18:59:22,964 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-12 18:59:22,964 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-12 18:59:23,069 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1022849511:84 , srv1027581002:84 , srv1039379083:84 , srv1061814457:84 , srv1078128795:84 , srv1094763173:84 , srv1108498195:84 , srv1115158366:84 , srv1124791288:84 , srv1145351267:84 , srv1159081130:84 , srv1200065582:84 , srv1236042682:84 , srv1267673633:84 , srv1319497590:84 , srv133933452:84 , srv1359463195:84 , srv1366801224:84 , srv1369234285:84 , srv1376993714:84 , srv1414837355:84 , srv1444920895:84 , srv1481212396:84 , srv1560891652:84 , srv1571022384:84 , srv1589507933:84 , srv1591597488:84 , srv1595485111:84 , srv1677821424:84 , srv1691280030:84 , srv1697523424:84 , srv1714893629:84 , srv1735369491:84 , srv1761280101:84 , srv1780137468:84 , srv1796531854:84 , srv1800909874:84 , srv1835588756:84 , srv1845022367:84 , srv1849891048:84 , srv1851537104:84 , srv1905145843:84 , srv1922256129:84 , srv1939236319:84 , srv1957818227:84 , srv1973585296:84 , srv1977509575:84 , srv1991058582:84 , srv1994625836:84 , srv2037610151:84 , srv205085364:84 , srv2113182436:84 , srv2119073991:84 , srv214931852:84 , srv262484820:84 , srv319416258:84 , srv319520302:84 , srv344456966:84 , srv356154356:84 , srv360647981:84 , srv37362296:84 , srv385611916:84 , srv387210422:84 , srv409179513:84 , srv411115771:84 , srv460505761:84 , srv461552813:84 , srv481250418:84 , srv49912557:84 , srv504668087:84 , srv505629427:84 , srv513028414:84 , srv53058047:84 , srv533566039:84 , srv53851864:84 , srv564922287:84 , srv584745909:84 , srv58605794:84 , srv597993706:84 , srv624219692:84 , srv628367744:84 , srv655988435:84 , srv66731110:84 , srv674009055:84 , srv707165893:84 , srv708081594:84 , srv719441673:84 , srv754254579:84 , srv785098505:84 , srv801481250:84 , srv836563379:84 , srv85134130:84 , srv85382276:84 , srv88151366:84 , srv888327501:84 , srv893388380:84 , srv928693750:84 , srv949957987:84 , srv983780115:84 , srv1640065629:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-12 18:59:23,076 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 18:59:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1159081130=10, srv344456966=58, srv504668087=70, srv1078128795=4, srv1027581002=1, srv2119073991=53, srv655988435=82, srv707165893=85, srv1376993714=19, srv385611916=62, srv66731110=83, srv262484820=55, srv1108498195=6, srv85382276=93, srv387210422=63, srv133933452=15, srv505629427=71, srv1589507933=25, srv564922287=76, srv1640065629=28, srv88151366=94, srv481250418=68, srv628367744=81, srv461552813=67, srv1735369491=33, srv1922256129=43, srv1973585296=46, srv1366801224=17, srv624219692=80, srv1061814457=3, srv1094763173=5, srv928693750=97, srv949957987=98, srv319520302=57, srv1444920895=21, srv1714893629=32, srv1115158366=7, srv409179513=64, srv1022849511=0, srv1796531854=36, srv2113182436=52, srv53058047=73, srv356154356=59, srv460505761=66, srv1845022367=39, srv1571022384=24, srv1991058582=48, srv893388380=96, srv1039379083=2, srv1124791288=8, srv1691280030=30, srv513028414=72, srv597993706=79, srv1851537104=41, srv360647981=60, srv674009055=84, srv1905145843=42, srv58605794=78, srv754254579=88, srv205085364=51, srv1319497590=14, srv1267673633=13, srv1369234285=18, srv1560891652=23, srv1414837355=20, srv85134130=92, srv411115771=65, srv2037610151=50, srv584745909=77, srv319416258=56, srv1957818227=45, srv888327501=95, srv1236042682=12, srv1939236319=44, srv1994625836=49, srv708081594=86, srv1849891048=40, srv1481212396=22, srv719441673=87, srv1359463195=16, srv1697523424=31, srv53851864=75, srv1780137468=35, srv836563379=91, srv983780115=99, srv1677821424=29, srv1591597488=26, srv801481250=90, srv1835588756=38, srv1761280101=34, srv1200065582=11, srv533566039=74, srv1977509575=47, srv1595485111=27, srv1145351267=9, srv785098505=89, srv49912557=69, srv214931852=54, srv37362296=61, srv1800909874=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 18:59:23,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 18:59:23,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 18:59:23,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 18:59:23,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 18:59:23,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 18:59:23,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 18:59:23,157 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 18:59:23,158 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 18:59:23,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 18:59:23,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 18:59:23,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 18:59:23,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 18:59:23,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 18:59:23,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 18:59:23,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 18:59:23,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 18:59:23,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 18:59:23,323 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:23,323 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 18:59:23,324 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 18:59:23,727 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 541 ms to try 5000 different iterations.  Found a solution that moves 4659 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1822849102666752. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03998347332860016, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05529230558802063, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5176666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03998347332860016, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17783333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24116666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:23,736 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1697523424:81 , srv1691280030:82 , srv37362296:82 , srv411115771:82 , srv481250418:82 , srv655988435:82 , srv983780115:82 , srv1022849511:83 , srv1159081130:83 , srv1267673633:83 , srv1560891652:83 , srv1957818227:83 , srv1977509575:83 , srv205085364:83 , srv319520302:83 , srv385611916:83 , srv460505761:83 , srv49912557:83 , srv597993706:83 , srv1236042682:84 , srv1366801224:84 , srv1444920895:84 , srv1735369491:84 , srv1800909874:84 , srv505629427:84 , srv53058047:84 , srv533566039:84 , srv564922287:84 , srv754254579:84 , srv85382276:84 , srv1115158366:85 , srv1589507933:85 , srv1835588756:85 , srv1922256129:85 , srv1994625836:85 , srv2119073991:85 , srv319416258:85 , srv344456966:85 , srv708081594:85 , srv1039379083:86 , srv1078128795:86 , srv1108498195:86 , srv1124791288:86 , srv1200065582:86 , srv133933452:86 , srv1359463195:86 , srv1595485111:86 , srv1714893629:86 , srv2037610151:86 , srv387210422:86 , srv461552813:86 , srv801481250:86 , srv88151366:86 , srv888327501:86 , srv1061814457:87 , srv1145351267:87 , srv1376993714:87 , srv1571022384:87 , srv1849891048:87 , srv1973585296:87 , srv214931852:87 , srv504668087:87 , srv628367744:87 , srv674009055:87 , srv1027581002:88 , srv1319497590:88 , srv1481212396:88 , srv1761280101:88 , srv1796531854:88 , srv1851537104:88 , srv1905145843:88 , srv356154356:88 , srv584745909:88 , srv624219692:88 , srv707165893:88 , srv85134130:88 , srv949957987:88 , srv1094763173:89 , srv1369234285:89 , srv1591597488:89 , srv1991058582:89 , srv262484820:89 , srv513028414:89 , srv53851864:89 , srv785098505:89 , srv836563379:89 , srv928693750:89 , srv1414837355:90 , srv1677821424:90 , srv1780137468:90 , srv2113182436:90 , srv1939236319:91 , srv58605794:91 , srv66731110:91 , srv409179513:92 , srv719441673:92 , srv1845022367:94 , srv360647981:94 , srv893388380:98 , srv1640065629:445 }
2023-07-12 18:59:23,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 18:59:23,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1159081130=10, srv344456966=58, srv504668087=70, srv1078128795=4, srv1027581002=1, srv2119073991=53, srv655988435=82, srv707165893=85, srv1376993714=19, srv385611916=62, srv66731110=83, srv262484820=55, srv1108498195=6, srv85382276=93, srv387210422=63, srv133933452=15, srv505629427=71, srv1589507933=25, srv564922287=76, srv1640065629=28, srv88151366=94, srv481250418=68, srv628367744=81, srv461552813=67, srv1735369491=33, srv1922256129=43, srv1973585296=46, srv1366801224=17, srv624219692=80, srv1061814457=3, srv1094763173=5, srv928693750=97, srv949957987=98, srv319520302=57, srv1444920895=21, srv1714893629=32, srv1115158366=7, srv409179513=64, srv1022849511=0, srv1796531854=36, srv2113182436=52, srv53058047=73, srv356154356=59, srv460505761=66, srv1845022367=39, srv1571022384=24, srv1991058582=48, srv893388380=96, srv1039379083=2, srv1124791288=8, srv1691280030=30, srv513028414=72, srv597993706=79, srv1851537104=41, srv360647981=60, srv674009055=84, srv1905145843=42, srv58605794=78, srv754254579=88, srv205085364=51, srv1319497590=14, srv1267673633=13, srv1369234285=18, srv1560891652=23, srv1414837355=20, srv85134130=92, srv411115771=65, srv2037610151=50, srv584745909=77, srv319416258=56, srv1957818227=45, srv888327501=95, srv1236042682=12, srv1939236319=44, srv1994625836=49, srv708081594=86, srv1849891048=40, srv1481212396=22, srv719441673=87, srv1359463195=16, srv1697523424=31, srv53851864=75, srv1780137468=35, srv836563379=91, srv983780115=99, srv1677821424=29, srv1591597488=26, srv801481250=90, srv1835588756=38, srv1761280101=34, srv1200065582=11, srv533566039=74, srv1977509575=47, srv1595485111=27, srv1145351267=9, srv785098505=89, srv49912557=69, srv214931852=54, srv37362296=61, srv1800909874=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 18:59:23,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 18:59:23,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 18:59:23,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 18:59:23,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 18:59:23,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 18:59:23,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 18:59:23,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 18:59:23,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 18:59:23,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 18:59:23,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 18:59:23,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 18:59:23,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03998347332860016, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05529230558802063, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03998347332860016, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17783333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24116666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:23,852 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 18:59:23,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18225228725140952, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03998347332860016, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05529230558802063, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03998347332860016, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17783333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24116666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 18:59:24,034 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 262 ms to try 5000 different iterations.  Found a solution that moves 2628 regions; Going from a computed imbalance of 0.18225228725140952 to a new imbalance of 0.014576190372342023. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022161533290822284); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03202482202275643, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.292, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022161533290822284); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,042 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv893388380:74 , srv360647981:76 , srv1845022367:79 , srv409179513:79 , srv719441673:79 , srv58605794:80 , srv1414837355:81 , srv2113182436:81 , srv1677821424:82 , srv1780137468:82 , srv1939236319:82 , srv1991058582:82 , srv785098505:82 , srv836563379:82 , srv1094763173:83 , srv1905145843:83 , srv53851864:83 , srv624219692:83 , srv66731110:83 , srv1027581002:84 , srv1039379083:84 , srv133933452:84 , srv1369234285:84 , srv1481212396:84 , srv1591597488:84 , srv1714893629:84 , srv1973585296:84 , srv262484820:84 , srv85134130:84 , srv88151366:84 , srv928693750:84 , srv1061814457:85 , srv1108498195:85 , srv1145351267:85 , srv1319497590:85 , srv1359463195:85 , srv1571022384:85 , srv1589507933:85 , srv1761280101:85 , srv1849891048:85 , srv1851537104:85 , srv1994625836:85 , srv2037610151:85 , srv214931852:85 , srv387210422:85 , srv504668087:85 , srv513028414:85 , srv584745909:85 , srv628367744:85 , srv707165893:85 , srv949957987:85 , srv1115158366:86 , srv1124791288:86 , srv1236042682:86 , srv1267673633:86 , srv1796531854:86 , srv1922256129:86 , srv356154356:86 , srv53058047:86 , srv708081594:86 , srv1078128795:87 , srv1200065582:87 , srv1595485111:87 , srv1735369491:87 , srv1800909874:87 , srv205085364:87 , srv2119073991:87 , srv461552813:87 , srv674009055:87 , srv754254579:87 , srv1022849511:88 , srv1376993714:88 , srv1957818227:88 , srv1977509575:88 , srv319416258:88 , srv344456966:88 , srv411115771:88 , srv49912557:88 , srv505629427:88 , srv564922287:88 , srv655988435:88 , srv801481250:88 , srv888327501:88 , srv1159081130:89 , srv1560891652:89 , srv1835588756:89 , srv385611916:89 , srv597993706:89 , srv983780115:89 , srv1691280030:90 , srv460505761:90 , srv533566039:90 , srv85382276:90 , srv1366801224:91 , srv319520302:91 , srv37362296:91 , srv481250418:91 , srv1444920895:92 , srv1697523424:92 , srv1640065629:526 }
2023-07-12 18:59:24,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 18:59:24,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1159081130=10, srv344456966=58, srv504668087=70, srv1078128795=4, srv1027581002=1, srv2119073991=53, srv655988435=82, srv707165893=85, srv1376993714=19, srv385611916=62, srv66731110=83, srv262484820=55, srv1108498195=6, srv85382276=93, srv387210422=63, srv133933452=15, srv505629427=71, srv1589507933=25, srv564922287=76, srv1640065629=28, srv88151366=94, srv481250418=68, srv628367744=81, srv461552813=67, srv1735369491=33, srv1922256129=43, srv1973585296=46, srv1366801224=17, srv624219692=80, srv1061814457=3, srv1094763173=5, srv928693750=97, srv949957987=98, srv319520302=57, srv1444920895=21, srv1714893629=32, srv1115158366=7, srv409179513=64, srv1022849511=0, srv1796531854=36, srv2113182436=52, srv53058047=73, srv356154356=59, srv460505761=66, srv1845022367=39, srv1571022384=24, srv1991058582=48, srv893388380=96, srv1039379083=2, srv1124791288=8, srv1691280030=30, srv513028414=72, srv597993706=79, srv1851537104=41, srv360647981=60, srv674009055=84, srv1905145843=42, srv58605794=78, srv754254579=88, srv205085364=51, srv1319497590=14, srv1267673633=13, srv1369234285=18, srv1560891652=23, srv1414837355=20, srv85134130=92, srv411115771=65, srv2037610151=50, srv584745909=77, srv319416258=56, srv1957818227=45, srv888327501=95, srv1236042682=12, srv1939236319=44, srv1994625836=49, srv708081594=86, srv1849891048=40, srv1481212396=22, srv719441673=87, srv1359463195=16, srv1697523424=31, srv53851864=75, srv1780137468=35, srv836563379=91, srv983780115=99, srv1677821424=29, srv1591597488=26, srv801481250=90, srv1835588756=38, srv1761280101=34, srv1200065582=11, srv533566039=74, srv1977509575=47, srv1595485111=27, srv1145351267=9, srv785098505=89, srv49912557=69, srv214931852=54, srv37362296=61, srv1800909874=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 18:59:24,060 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 18:59:24,061 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 18:59:24,062 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 18:59:24,063 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 18:59:24,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 18:59:24,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 18:59:24,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 18:59:24,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 18:59:24,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 18:59:24,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 18:59:24,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 18:59:24,067 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 18:59:24,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022161533290822284); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03202482202275643, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022161533290822284); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,151 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 18:59:24,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014557788723035685, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022161533290822284); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03202482202275643, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022161533290822284); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04558333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 18:59:24,309 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 232 ms to try 5000 different iterations.  Found a solution that moves 987 regions; Going from a computed imbalance of 0.014557788723035685 to a new imbalance of 6.957598485154196E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005311120972618093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009014267279726507); MoveCostFunction : (multiplier=7.0, imbalance=0.10966666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005311120972618093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,311 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1444920895:81 , srv1366801224:82 , srv1376993714:82 , srv1796531854:83 , srv533566039:83 , srv674009055:83 , srv888327501:83 , srv1078128795:84 , srv1200065582:84 , srv1319497590:84 , srv1697523424:84 , srv1835588756:84 , srv262484820:84 , srv319416258:84 , srv344456966:84 , srv356154356:84 , srv461552813:84 , srv481250418:84 , srv513028414:84 , srv66731110:84 , srv707165893:84 , srv801481250:84 , srv85382276:84 , srv1061814457:85 , srv1369234285:85 , srv1560891652:85 , srv1571022384:85 , srv1595485111:85 , srv1761280101:85 , srv1780137468:85 , srv1845022367:85 , srv1849891048:85 , srv1939236319:85 , srv2119073991:85 , srv319520302:85 , srv37362296:85 , srv385611916:85 , srv460505761:85 , srv564922287:85 , srv584745909:85 , srv597993706:85 , srv628367744:85 , srv928693750:85 , srv949957987:85 , srv1022849511:86 , srv1027581002:86 , srv1094763173:86 , srv1108498195:86 , srv1115158366:86 , srv1124791288:86 , srv1145351267:86 , srv1159081130:86 , srv1481212396:86 , srv1591597488:86 , srv1677821424:86 , srv1691280030:86 , srv1735369491:86 , srv1851537104:86 , srv1957818227:86 , srv1991058582:86 , srv2037610151:86 , srv214931852:86 , srv387210422:86 , srv409179513:86 , srv49912557:86 , srv504668087:86 , srv505629427:86 , srv53851864:86 , srv58605794:86 , srv624219692:86 , srv719441673:86 , srv85134130:86 , srv893388380:86 , srv983780115:86 , srv1039379083:87 , srv1236042682:87 , srv1359463195:87 , srv1414837355:87 , srv1714893629:87 , srv1800909874:87 , srv1905145843:87 , srv1922256129:87 , srv1973585296:87 , srv1977509575:87 , srv1994625836:87 , srv205085364:87 , srv2113182436:87 , srv411115771:87 , srv53058047:87 , srv708081594:87 , srv754254579:87 , srv785098505:87 , srv836563379:87 , srv88151366:87 , srv1267673633:88 , srv133933452:88 , srv1589507933:88 , srv360647981:88 , srv655988435:88 , srv1640065629:534 }
2023-07-12 18:59:24,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 18:59:24,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1159081130=10, srv344456966=58, srv504668087=70, srv1078128795=4, srv1027581002=1, srv2119073991=53, srv655988435=82, srv707165893=85, srv1376993714=19, srv385611916=62, srv66731110=83, srv262484820=55, srv1108498195=6, srv85382276=93, srv387210422=63, srv133933452=15, srv505629427=71, srv1589507933=25, srv564922287=76, srv1640065629=28, srv88151366=94, srv481250418=68, srv628367744=81, srv461552813=67, srv1735369491=33, srv1922256129=43, srv1973585296=46, srv1366801224=17, srv624219692=80, srv1061814457=3, srv1094763173=5, srv928693750=97, srv949957987=98, srv319520302=57, srv1444920895=21, srv1714893629=32, srv1115158366=7, srv409179513=64, srv1022849511=0, srv1796531854=36, srv2113182436=52, srv53058047=73, srv356154356=59, srv460505761=66, srv1845022367=39, srv1571022384=24, srv1991058582=48, srv893388380=96, srv1039379083=2, srv1124791288=8, srv1691280030=30, srv513028414=72, srv597993706=79, srv1851537104=41, srv360647981=60, srv674009055=84, srv1905145843=42, srv58605794=78, srv754254579=88, srv205085364=51, srv1319497590=14, srv1267673633=13, srv1369234285=18, srv1560891652=23, srv1414837355=20, srv85134130=92, srv411115771=65, srv2037610151=50, srv584745909=77, srv319416258=56, srv1957818227=45, srv888327501=95, srv1236042682=12, srv1939236319=44, srv1994625836=49, srv708081594=86, srv1849891048=40, srv1481212396=22, srv719441673=87, srv1359463195=16, srv1697523424=31, srv53851864=75, srv1780137468=35, srv836563379=91, srv983780115=99, srv1677821424=29, srv1591597488=26, srv801481250=90, srv1835588756=38, srv1761280101=34, srv1200065582=11, srv533566039=74, srv1977509575=47, srv1595485111=27, srv1145351267=9, srv785098505=89, srv49912557=69, srv214931852=54, srv37362296=61, srv1800909874=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 18:59:24,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 18:59:24,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 18:59:24,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 18:59:24,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 18:59:24,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 18:59:24,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 18:59:24,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 18:59:24,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 18:59:24,411 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005311120972618093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009014267279726507); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005311120972618093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,411 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-12 18:59:24,411 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.888487268010534E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005311120972618093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009014267279726507); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005311120972618093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-12 18:59:24,651 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 313 ms to try 5000 different iterations.  Found a solution that moves 258 regions; Going from a computed imbalance of 6.888487268010534E-4 to a new imbalance of 2.050841734611288E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.028666666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,651 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1359463195:83 , srv1414837355:83 , srv1591597488:83 , srv1851537104:83 , srv1939236319:83 , srv319520302:83 , srv1027581002:84 , srv1094763173:84 , srv1124791288:84 , srv1145351267:84 , srv1159081130:84 , srv1369234285:84 , srv1481212396:84 , srv1589507933:84 , srv1595485111:84 , srv1677821424:84 , srv1796531854:84 , srv1800909874:84 , srv1835588756:84 , srv1845022367:84 , srv1905145843:84 , srv1922256129:84 , srv1973585296:84 , srv1977509575:84 , srv1991058582:84 , srv2113182436:84 , srv214931852:84 , srv356154356:84 , srv360647981:84 , srv37362296:84 , srv409179513:84 , srv460505761:84 , srv504668087:84 , srv505629427:84 , srv513028414:84 , srv564922287:84 , srv584745909:84 , srv58605794:84 , srv655988435:84 , srv66731110:84 , srv754254579:84 , srv801481250:84 , srv836563379:84 , srv85134130:84 , srv85382276:84 , srv893388380:84 , srv928693750:84 , srv949957987:84 , srv1022849511:85 , srv1039379083:85 , srv1061814457:85 , srv1078128795:85 , srv1108498195:85 , srv1115158366:85 , srv1200065582:85 , srv1236042682:85 , srv1267673633:85 , srv1319497590:85 , srv133933452:85 , srv1366801224:85 , srv1376993714:85 , srv1444920895:85 , srv1560891652:85 , srv1571022384:85 , srv1691280030:85 , srv1697523424:85 , srv1714893629:85 , srv1735369491:85 , srv1761280101:85 , srv1780137468:85 , srv1849891048:85 , srv1957818227:85 , srv1994625836:85 , srv2037610151:85 , srv2119073991:85 , srv262484820:85 , srv319416258:85 , srv344456966:85 , srv387210422:85 , srv411115771:85 , srv461552813:85 , srv481250418:85 , srv49912557:85 , srv53058047:85 , srv533566039:85 , srv53851864:85 , srv597993706:85 , srv624219692:85 , srv628367744:85 , srv674009055:85 , srv707165893:85 , srv708081594:85 , srv719441673:85 , srv785098505:85 , srv88151366:85 , srv888327501:85 , srv983780115:85 , srv205085364:86 , srv385611916:86 , srv1640065629:637 }
2023-07-12 18:59:24,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 18:59:24,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1159081130=10, srv344456966=58, srv504668087=70, srv1078128795=4, srv1027581002=1, srv2119073991=53, srv655988435=82, srv707165893=85, srv1376993714=19, srv385611916=62, srv66731110=83, srv262484820=55, srv1108498195=6, srv85382276=93, srv387210422=63, srv133933452=15, srv505629427=71, srv1589507933=25, srv564922287=76, srv1640065629=28, srv88151366=94, srv481250418=68, srv628367744=81, srv461552813=67, srv1735369491=33, srv1922256129=43, srv1973585296=46, srv1366801224=17, srv624219692=80, srv1061814457=3, srv1094763173=5, srv928693750=97, srv949957987=98, srv319520302=57, srv1444920895=21, srv1714893629=32, srv1115158366=7, srv409179513=64, srv1022849511=0, srv1796531854=36, srv2113182436=52, srv53058047=73, srv356154356=59, srv460505761=66, srv1845022367=39, srv1571022384=24, srv1991058582=48, srv893388380=96, srv1039379083=2, srv1124791288=8, srv1691280030=30, srv513028414=72, srv597993706=79, srv1851537104=41, srv360647981=60, srv674009055=84, srv1905145843=42, srv58605794=78, srv754254579=88, srv205085364=51, srv1319497590=14, srv1267673633=13, srv1369234285=18, srv1560891652=23, srv1414837355=20, srv85134130=92, srv411115771=65, srv2037610151=50, srv584745909=77, srv319416258=56, srv1957818227=45, srv888327501=95, srv1236042682=12, srv1939236319=44, srv1994625836=49, srv708081594=86, srv1849891048=40, srv1481212396=22, srv719441673=87, srv1359463195=16, srv1697523424=31, srv53851864=75, srv1780137468=35, srv836563379=91, srv983780115=99, srv1677821424=29, srv1591597488=26, srv801481250=90, srv1835588756=38, srv1761280101=34, srv1200065582=11, srv533566039=74, srv1977509575=47, srv1595485111=27, srv1145351267=9, srv785098505=89, srv49912557=69, srv214931852=54, srv37362296=61, srv1800909874=37} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 18:59:24,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 18:59:24,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-12 18:59:24,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-12 18:59:24,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-12 18:59:24,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-12 18:59:24,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-12 18:59:24,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-12 18:59:24,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-12 18:59:24,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-12 18:59:24,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.8701862733846913E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,781 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1359463195:83 , srv1414837355:83 , srv1591597488:83 , srv1851537104:83 , srv1939236319:83 , srv319520302:83 , srv1027581002:84 , srv1094763173:84 , srv1124791288:84 , srv1145351267:84 , srv1159081130:84 , srv1369234285:84 , srv1481212396:84 , srv1589507933:84 , srv1595485111:84 , srv1677821424:84 , srv1796531854:84 , srv1800909874:84 , srv1835588756:84 , srv1845022367:84 , srv1905145843:84 , srv1922256129:84 , srv1973585296:84 , srv1977509575:84 , srv1991058582:84 , srv2113182436:84 , srv214931852:84 , srv356154356:84 , srv360647981:84 , srv37362296:84 , srv409179513:84 , srv460505761:84 , srv504668087:84 , srv505629427:84 , srv513028414:84 , srv564922287:84 , srv584745909:84 , srv58605794:84 , srv655988435:84 , srv66731110:84 , srv754254579:84 , srv801481250:84 , srv836563379:84 , srv85134130:84 , srv85382276:84 , srv893388380:84 , srv928693750:84 , srv949957987:84 , srv1022849511:85 , srv1039379083:85 , srv1061814457:85 , srv1078128795:85 , srv1108498195:85 , srv1115158366:85 , srv1200065582:85 , srv1236042682:85 , srv1267673633:85 , srv1319497590:85 , srv133933452:85 , srv1366801224:85 , srv1376993714:85 , srv1444920895:85 , srv1560891652:85 , srv1571022384:85 , srv1691280030:85 , srv1697523424:85 , srv1714893629:85 , srv1735369491:85 , srv1761280101:85 , srv1780137468:85 , srv1849891048:85 , srv1957818227:85 , srv1994625836:85 , srv2037610151:85 , srv2119073991:85 , srv262484820:85 , srv319416258:85 , srv344456966:85 , srv387210422:85 , srv411115771:85 , srv461552813:85 , srv481250418:85 , srv49912557:85 , srv53058047:85 , srv533566039:85 , srv53851864:85 , srv597993706:85 , srv624219692:85 , srv628367744:85 , srv674009055:85 , srv707165893:85 , srv708081594:85 , srv719441673:85 , srv785098505:85 , srv88151366:85 , srv888327501:85 , srv983780115:85 , srv205085364:86 , srv385611916:86 , srv1640065629:637 }
2023-07-12 18:59:24,803 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 144) - SystemLoadAverage LEAK? -, ProcessCount=164 (was 164), AvailableMemoryMB=6218 (was 6704)
2023-07-12 18:59:24,810 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=173, ProcessCount=164, AvailableMemoryMB=6218
2023-07-12 18:59:24,811 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-12 18:59:24,811 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-12 18:59:24,819 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1149711346:84 , srv1179782887:84 , srv123845348:84 , srv1244205110:84 , srv1308039250:84 , srv1334352495:84 , srv1387819083:84 , srv1444990809:84 , srv1448881448:84 , srv1484884420:84 , srv1486247564:84 , srv1622635056:84 , srv1707411783:84 , srv1718564211:84 , srv1845802806:84 , srv1923871212:84 , srv224546658:84 , srv355968161:84 , srv43773473:84 , srv491007828:84 , srv641062661:84 , srv69986317:84 , srv716675680:84 , srv730520636:84 , srv776900642:84 , srv782693123:84 , srv843172707:84 , srv86840322:84 , srv893081093:84 , srv1765940983:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-12 18:59:24,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 18:59:24,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv730520636=24, srv43773473=19, srv1387819083=6, srv1486247564=10, srv1334352495=5, srv1622635056=11, srv123845348=2, srv355968161=18, srv1484884420=9, srv69986317=22, srv1244205110=3, srv1923871212=16, srv641062661=21, srv1707411783=12, srv491007828=20, srv843172707=27, srv1444990809=7, srv224546658=17, srv1718564211=13, srv1308039250=4, srv86840322=28, srv1448881448=8, srv1179782887=1, srv893081093=29, srv782693123=26, srv1149711346=0, srv1845802806=15, srv776900642=25, srv716675680=23, srv1765940983=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 18:59:24,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 18:59:24,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 18:59:24,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 18:59:24,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 18:59:24,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 18:59:24,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 18:59:24,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 18:59:24,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-12 18:59:24,844 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.085494148468309, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 18:59:24,845 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912462432534462, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.085494148468309, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-12 19:01:24,833 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 10059178 different iterations.  Found a solution that moves 1836 regions; Going from a computed imbalance of 0.9912462432534462 to a new imbalance of 4.2853155918867096E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.68, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 19:01:24,838 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1149711346:90 , srv1179782887:90 , srv123845348:90 , srv1244205110:90 , srv1308039250:90 , srv1334352495:90 , srv1387819083:90 , srv1444990809:90 , srv1448881448:90 , srv1484884420:90 , srv1486247564:90 , srv1622635056:90 , srv1707411783:90 , srv1718564211:90 , srv1765940983:90 , srv1845802806:90 , srv1923871212:90 , srv224546658:90 , srv355968161:90 , srv43773473:90 , srv491007828:90 , srv641062661:90 , srv69986317:90 , srv716675680:90 , srv730520636:90 , srv776900642:90 , srv782693123:90 , srv843172707:90 , srv86840322:90 , srv893081093:90 }
2023-07-12 19:01:24,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-12 19:01:24,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv730520636=24, srv43773473=19, srv1387819083=6, srv1486247564=10, srv1334352495=5, srv1622635056=11, srv123845348=2, srv355968161=18, srv1484884420=9, srv69986317=22, srv1244205110=3, srv1923871212=16, srv641062661=21, srv1707411783=12, srv491007828=20, srv843172707=27, srv1444990809=7, srv224546658=17, srv1718564211=13, srv1308039250=4, srv86840322=28, srv1448881448=8, srv1179782887=1, srv893081093=29, srv782693123=26, srv1149711346=0, srv1845802806=15, srv776900642=25, srv716675680=23, srv1765940983=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-12 19:01:24,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-12 19:01:24,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-12 19:01:24,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-12 19:01:24,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-12 19:01:24,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-12 19:01:24,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-12 19:01:24,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-12 19:01:24,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-12 19:01:24,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-12 19:01:24,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-12 19:01:24,853 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-12 19:01:24,853 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1149711346:90 , srv1179782887:90 , srv123845348:90 , srv1244205110:90 , srv1308039250:90 , srv1334352495:90 , srv1387819083:90 , srv1444990809:90 , srv1448881448:90 , srv1484884420:90 , srv1486247564:90 , srv1622635056:90 , srv1707411783:90 , srv1718564211:90 , srv1765940983:90 , srv1845802806:90 , srv1923871212:90 , srv224546658:90 , srv355968161:90 , srv43773473:90 , srv491007828:90 , srv641062661:90 , srv69986317:90 , srv716675680:90 , srv730520636:90 , srv776900642:90 , srv782693123:90 , srv843172707:90 , srv86840322:90 , srv893081093:90 }
2023-07-12 19:01:24,864 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=125 (was 173), ProcessCount=165 (was 164) - ProcessCount LEAK? -, AvailableMemoryMB=5994 (was 6218)