2024-02-12 00:53:11,842 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-02-12 00:53:12,145 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-02-12 00:53:12,254 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-02-12 00:53:12,316 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-12 00:53:12,322 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-12 00:53:12,335 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=286, ProcessCount=170, AvailableMemoryMB=3311
2024-02-12 00:53:12,338 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-12 00:53:12,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-02-12 00:53:12,473 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv104188442:84 , srv1055849666:84 , srv1092582754:84 , srv112335858:84 , srv1192246041:84 , srv1208196109:84 , srv1216679351:84 , srv1235553798:84 , srv1236580324:84 , srv1290216126:84 , srv1295097647:84 , srv1351181789:84 , srv1352464290:84 , srv1374907824:84 , srv1388212027:84 , srv1432914081:84 , srv1452980935:84 , srv1522664035:84 , srv1540728694:84 , srv1550884118:84 , srv1558303981:84 , srv1578680736:84 , srv164853438:84 , srv1657661728:84 , srv1738867983:84 , srv1740222354:84 , srv1746708996:84 , srv1764388874:84 , srv1777308603:84 , srv183225762:84 , srv1833915426:84 , srv1858723294:84 , srv1868205806:84 , srv1870643237:84 , srv1885146682:84 , srv1889106797:84 , srv1913015804:84 , srv1926847855:84 , srv1964256749:84 , srv1970947927:84 , srv2003381805:84 , srv2005547657:84 , srv2013957265:84 , srv2015456540:84 , srv2031786725:84 , srv205954963:84 , srv2101091745:84 , srv2136308133:84 , srv2147202144:84 , srv231171549:84 , srv233096350:84 , srv239680214:84 , srv253837984:84 , srv256516198:84 , srv256549025:84 , srv262322822:84 , srv280032993:84 , srv305726797:84 , srv349537603:84 , srv355393414:84 , srv35616092:84 , srv365082087:84 , srv381564646:84 , srv383780164:84 , srv397471758:84 , srv41184645:84 , srv493454760:84 , srv494445693:84 , srv528039816:84 , srv535552269:84 , srv537610337:84 , srv54112261:84 , srv561225002:84 , srv580566078:84 , srv591170030:84 , srv613528067:84 , srv634567242:84 , srv669744420:84 , srv755134548:84 , srv771333179:84 , srv776217962:84 , srv780070327:84 , srv804515424:84 , srv806394446:84 , srv807240750:84 , srv82933296:84 , srv846394236:84 , srv847346856:84 , srv8680517:84 , srv882409562:84 , srv88573834:84 , srv901100843:84 , srv912415846:84 , srv920970928:84 , srv929135658:84 , srv930442462:84 , srv935775628:84 , srv938367911:84 , srv949450176:84 , srv518424428:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-02-12 00:53:12,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:12,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1657661728=23, srv365082087=61, srv535552269=70, srv2015456540=43, srv771333179=80, srv381564646=62, srv494445693=67, srv183225762=29, srv1885146682=34, srv493454760=66, srv35616092=60, srv1290216126=9, srv1558303981=20, srv591170030=75, srv1964256749=38, srv938367911=98, srv239680214=51, srv2101091745=46, srv82933296=86, srv280032993=56, srv1913015804=36, srv929135658=95, srv164853438=22, srv88573834=91, srv1870643237=33, srv528039816=69, srv1550884118=19, srv669744420=78, srv1432914081=15, srv1578680736=21, srv2005547657=41, srv1192246041=4, srv41184645=65, srv231171549=49, srv804515424=83, srv920970928=94, srv2003381805=40, srv1777308603=28, srv1452980935=16, srv104188442=0, srv1295097647=10, srv1926847855=37, srv807240750=85, srv634567242=77, srv205954963=45, srv847346856=88, srv780070327=82, srv2136308133=47, srv1092582754=2, srv1216679351=6, srv806394446=84, srv1740222354=25, srv262322822=55, srv537610337=71, srv1868205806=32, srv397471758=64, srv1889106797=35, srv1235553798=7, srv1055849666=1, srv1746708996=26, srv1738867983=24, srv613528067=76, srv233096350=50, srv755134548=79, srv1208196109=5, srv912415846=93, srv54112261=72, srv935775628=97, srv1352464290=12, srv1858723294=31, srv2013957265=42, srv253837984=52, srv1764388874=27, srv882409562=90, srv256549025=54, srv580566078=74, srv1374907824=13, srv1970947927=39, srv355393414=59, srv1833915426=30, srv930442462=96, srv1351181789=11, srv561225002=73, srv1388212027=14, srv901100843=92, srv349537603=58, srv776217962=81, srv2147202144=48, srv305726797=57, srv846394236=87, srv383780164=63, srv949450176=99, srv1540728694=18, srv1236580324=8, srv112335858=3, srv256516198=53, srv518424428=68, srv1522664035=17, srv2031786725=44, srv8680517=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:12,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:12,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:12,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:12,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:12,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-12 00:53:12,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-12 00:53:12,597 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-12 00:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-12 00:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-12 00:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-12 00:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-12 00:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-12 00:53:12,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-12 00:53:12,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-12 00:53:12,600 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-12 00:53:12,601 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-12 00:53:12,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-12 00:53:12,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-12 00:53:12,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-12 00:53:12,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-12 00:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-12 00:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-12 00:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-12 00:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-12 00:53:12,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-12 00:53:12,969 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:12,969 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-12 00:53:12,969 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-12 00:53:13,851 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1213 ms to try 5000 different iterations.  Found a solution that moves 4698 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17536661815305568. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039022537701748385, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05600024049972407, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.522, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039022537701748385, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17041666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:13,871 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1432914081:81 , srv1964256749:83 , srv591170030:83 , srv780070327:83 , srv1055849666:84 , srv1192246041:84 , srv1388212027:84 , srv1578680736:84 , srv1740222354:84 , srv1746708996:84 , srv1777308603:84 , srv239680214:84 , srv256516198:84 , srv807240750:84 , srv920970928:84 , srv938367911:84 , srv104188442:85 , srv1092582754:85 , srv1295097647:85 , srv1550884118:85 , srv1657661728:85 , srv1738867983:85 , srv2005547657:85 , srv2013957265:85 , srv2136308133:85 , srv233096350:85 , srv253837984:85 , srv349537603:85 , srv35616092:85 , srv365082087:85 , srv41184645:85 , srv535552269:85 , srv537610337:85 , srv847346856:85 , srv8680517:85 , srv882409562:85 , srv1216679351:86 , srv1235553798:86 , srv1351181789:86 , srv1374907824:86 , srv1764388874:86 , srv1868205806:86 , srv1870643237:86 , srv2003381805:86 , srv2015456540:86 , srv205954963:86 , srv231171549:86 , srv280032993:86 , srv494445693:86 , srv82933296:86 , srv846394236:86 , srv88573834:86 , srv901100843:86 , srv912415846:86 , srv935775628:86 , srv1236580324:87 , srv1290216126:87 , srv1352464290:87 , srv1858723294:87 , srv256549025:87 , srv305726797:87 , srv355393414:87 , srv397471758:87 , srv528039816:87 , srv561225002:87 , srv669744420:87 , srv771333179:87 , srv776217962:87 , srv806394446:87 , srv930442462:87 , srv1208196109:88 , srv1452980935:88 , srv1522664035:88 , srv183225762:88 , srv1833915426:88 , srv1885146682:88 , srv1913015804:88 , srv2101091745:88 , srv262322822:88 , srv493454760:88 , srv580566078:88 , srv613528067:88 , srv755134548:88 , srv949450176:88 , srv112335858:89 , srv1540728694:89 , srv1558303981:89 , srv1889106797:89 , srv2147202144:89 , srv383780164:89 , srv54112261:89 , srv804515424:89 , srv634567242:90 , srv164853438:91 , srv1926847855:91 , srv1970947927:91 , srv381564646:91 , srv929135658:93 , srv2031786725:95 , srv518424428:437 }
2024-02-12 00:53:13,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:13,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1657661728=23, srv365082087=61, srv535552269=70, srv2015456540=43, srv771333179=80, srv381564646=62, srv494445693=67, srv183225762=29, srv1885146682=34, srv493454760=66, srv35616092=60, srv1290216126=9, srv1558303981=20, srv591170030=75, srv1964256749=38, srv938367911=98, srv239680214=51, srv2101091745=46, srv82933296=86, srv280032993=56, srv1913015804=36, srv929135658=95, srv164853438=22, srv88573834=91, srv1870643237=33, srv528039816=69, srv1550884118=19, srv669744420=78, srv1432914081=15, srv1578680736=21, srv2005547657=41, srv1192246041=4, srv41184645=65, srv231171549=49, srv804515424=83, srv920970928=94, srv2003381805=40, srv1777308603=28, srv1452980935=16, srv104188442=0, srv1295097647=10, srv1926847855=37, srv807240750=85, srv634567242=77, srv205954963=45, srv847346856=88, srv780070327=82, srv2136308133=47, srv1092582754=2, srv1216679351=6, srv806394446=84, srv1740222354=25, srv262322822=55, srv537610337=71, srv1868205806=32, srv397471758=64, srv1889106797=35, srv1235553798=7, srv1055849666=1, srv1746708996=26, srv1738867983=24, srv613528067=76, srv233096350=50, srv755134548=79, srv1208196109=5, srv912415846=93, srv54112261=72, srv935775628=97, srv1352464290=12, srv1858723294=31, srv2013957265=42, srv253837984=52, srv1764388874=27, srv882409562=90, srv256549025=54, srv580566078=74, srv1374907824=13, srv1970947927=39, srv355393414=59, srv1833915426=30, srv930442462=96, srv1351181789=11, srv561225002=73, srv1388212027=14, srv901100843=92, srv349537603=58, srv776217962=81, srv2147202144=48, srv305726797=57, srv846394236=87, srv383780164=63, srv949450176=99, srv1540728694=18, srv1236580324=8, srv112335858=3, srv256516198=53, srv518424428=68, srv1522664035=17, srv2031786725=44, srv8680517=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:13,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:13,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:13,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:13,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:13,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:13,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:13,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:13,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:13,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:13,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-12 00:53:13,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-12 00:53:13,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-12 00:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-12 00:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-12 00:53:13,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-12 00:53:13,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-12 00:53:13,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-12 00:53:13,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-12 00:53:13,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-12 00:53:13,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-12 00:53:13,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-12 00:53:13,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-12 00:53:14,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039022537701748385, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05600024049972407, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039022537701748385, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17041666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:14,214 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-12 00:53:14,214 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17533372205395326, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039022537701748385, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05600024049972407, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039022537701748385, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17041666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2385, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-12 00:53:14,523 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 567 ms to try 5000 different iterations.  Found a solution that moves 2645 regions; Going from a computed imbalance of 0.17533372205395326 to a new imbalance of 0.014347568386649998. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0209292777626056); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03419426905074449, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29388888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0209292777626056); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04633333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:14,534 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2031786725:78 , srv929135658:80 , srv164853438:81 , srv381564646:81 , srv1208196109:82 , srv1452980935:82 , srv1540728694:82 , srv1558303981:82 , srv1889106797:82 , srv1926847855:82 , srv1970947927:82 , srv580566078:82 , srv804515424:82 , srv949450176:82 , srv112335858:83 , srv1858723294:83 , srv1913015804:83 , srv2147202144:83 , srv383780164:83 , srv493454760:83 , srv528039816:83 , srv613528067:83 , srv755134548:83 , srv1235553798:84 , srv1236580324:84 , srv1522664035:84 , srv183225762:84 , srv1833915426:84 , srv2101091745:84 , srv397471758:84 , srv54112261:84 , srv771333179:84 , srv1351181789:85 , srv2003381805:85 , srv231171549:85 , srv280032993:85 , srv305726797:85 , srv35616092:85 , srv634567242:85 , srv669744420:85 , srv806394446:85 , srv1657661728:86 , srv1738867983:86 , srv1740222354:86 , srv1885146682:86 , srv233096350:86 , srv253837984:86 , srv262322822:86 , srv494445693:86 , srv561225002:86 , srv776217962:86 , srv846394236:86 , srv882409562:86 , srv912415846:86 , srv930442462:86 , srv935775628:86 , srv104188442:87 , srv1295097647:87 , srv1352464290:87 , srv1374907824:87 , srv1388212027:87 , srv1550884118:87 , srv1746708996:87 , srv1868205806:87 , srv2005547657:87 , srv2015456540:87 , srv205954963:87 , srv2136308133:87 , srv239680214:87 , srv256549025:87 , srv349537603:87 , srv355393414:87 , srv780070327:87 , srv847346856:87 , srv8680517:87 , srv88573834:87 , srv901100843:87 , srv920970928:87 , srv938367911:87 , srv1055849666:88 , srv1092582754:88 , srv1192246041:88 , srv1216679351:88 , srv1290216126:88 , srv1764388874:88 , srv1870643237:88 , srv2013957265:88 , srv535552269:88 , srv537610337:88 , srv591170030:88 , srv807240750:88 , srv256516198:89 , srv41184645:89 , srv82933296:89 , srv1578680736:90 , srv1777308603:91 , srv1964256749:91 , srv365082087:91 , srv1432914081:92 , srv518424428:523 }
2024-02-12 00:53:14,538 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:14,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1657661728=23, srv365082087=61, srv535552269=70, srv2015456540=43, srv771333179=80, srv381564646=62, srv494445693=67, srv183225762=29, srv1885146682=34, srv493454760=66, srv35616092=60, srv1290216126=9, srv1558303981=20, srv591170030=75, srv1964256749=38, srv938367911=98, srv239680214=51, srv2101091745=46, srv82933296=86, srv280032993=56, srv1913015804=36, srv929135658=95, srv164853438=22, srv88573834=91, srv1870643237=33, srv528039816=69, srv1550884118=19, srv669744420=78, srv1432914081=15, srv1578680736=21, srv2005547657=41, srv1192246041=4, srv41184645=65, srv231171549=49, srv804515424=83, srv920970928=94, srv2003381805=40, srv1777308603=28, srv1452980935=16, srv104188442=0, srv1295097647=10, srv1926847855=37, srv807240750=85, srv634567242=77, srv205954963=45, srv847346856=88, srv780070327=82, srv2136308133=47, srv1092582754=2, srv1216679351=6, srv806394446=84, srv1740222354=25, srv262322822=55, srv537610337=71, srv1868205806=32, srv397471758=64, srv1889106797=35, srv1235553798=7, srv1055849666=1, srv1746708996=26, srv1738867983=24, srv613528067=76, srv233096350=50, srv755134548=79, srv1208196109=5, srv912415846=93, srv54112261=72, srv935775628=97, srv1352464290=12, srv1858723294=31, srv2013957265=42, srv253837984=52, srv1764388874=27, srv882409562=90, srv256549025=54, srv580566078=74, srv1374907824=13, srv1970947927=39, srv355393414=59, srv1833915426=30, srv930442462=96, srv1351181789=11, srv561225002=73, srv1388212027=14, srv901100843=92, srv349537603=58, srv776217962=81, srv2147202144=48, srv305726797=57, srv846394236=87, srv383780164=63, srv949450176=99, srv1540728694=18, srv1236580324=8, srv112335858=3, srv256516198=53, srv518424428=68, srv1522664035=17, srv2031786725=44, srv8680517=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:14,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:14,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:14,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-12 00:53:14,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-12 00:53:14,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-12 00:53:14,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-12 00:53:14,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-12 00:53:14,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-12 00:53:14,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-12 00:53:14,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-12 00:53:14,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-12 00:53:14,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-12 00:53:14,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-12 00:53:14,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-12 00:53:14,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-12 00:53:14,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-12 00:53:14,665 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0209292777626056); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03419426905074449, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0209292777626056); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04633333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:14,665 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-12 00:53:14,665 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014329047700799442, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0209292777626056); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03419426905074449, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0209292777626056); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04633333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-12 00:53:14,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 294 ms to try 5000 different iterations.  Found a solution that moves 992 regions; Going from a computed imbalance of 0.014329047700799442 to a new imbalance of 7.961287566164548E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005552187531265666); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007712569361139828); MoveCostFunction : (multiplier=7.0, imbalance=0.11022222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005552187531265666); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:14,871 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv365082087:81 , srv1777308603:82 , srv82933296:82 , srv1216679351:83 , srv1290216126:83 , srv1870643237:83 , srv1964256749:83 , srv355393414:83 , srv634567242:83 , srv1092582754:84 , srv1352464290:84 , srv1374907824:84 , srv1578680736:84 , srv1926847855:84 , srv1970947927:84 , srv2013957265:84 , srv256549025:84 , srv262322822:84 , srv41184645:84 , srv535552269:84 , srv537610337:84 , srv54112261:84 , srv561225002:84 , srv776217962:84 , srv88573834:84 , srv1192246041:85 , srv1295097647:85 , srv1522664035:85 , srv164853438:85 , srv1833915426:85 , srv1868205806:85 , srv1885146682:85 , srv2015456540:85 , srv2031786725:85 , srv205954963:85 , srv2136308133:85 , srv2147202144:85 , srv256516198:85 , srv305726797:85 , srv349537603:85 , srv381564646:85 , srv383780164:85 , srv669744420:85 , srv806394446:85 , srv847346856:85 , srv8680517:85 , srv901100843:85 , srv929135658:85 , srv930442462:85 , srv935775628:85 , srv104188442:86 , srv1055849666:86 , srv1236580324:86 , srv1351181789:86 , srv1432914081:86 , srv1550884118:86 , srv1746708996:86 , srv183225762:86 , srv1889106797:86 , srv2003381805:86 , srv2005547657:86 , srv2101091745:86 , srv233096350:86 , srv239680214:86 , srv253837984:86 , srv280032993:86 , srv493454760:86 , srv494445693:86 , srv591170030:86 , srv613528067:86 , srv771333179:86 , srv804515424:86 , srv807240750:86 , srv846394236:86 , srv882409562:86 , srv912415846:86 , srv920970928:86 , srv938367911:86 , srv112335858:87 , srv1235553798:87 , srv1388212027:87 , srv1452980935:87 , srv1540728694:87 , srv1558303981:87 , srv1657661728:87 , srv1738867983:87 , srv1764388874:87 , srv1858723294:87 , srv1913015804:87 , srv231171549:87 , srv528039816:87 , srv580566078:87 , srv755134548:87 , srv949450176:87 , srv1740222354:88 , srv35616092:88 , srv397471758:88 , srv780070327:88 , srv1208196109:89 , srv518424428:547 }
2024-02-12 00:53:14,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:14,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1657661728=23, srv365082087=61, srv535552269=70, srv2015456540=43, srv771333179=80, srv381564646=62, srv494445693=67, srv183225762=29, srv1885146682=34, srv493454760=66, srv35616092=60, srv1290216126=9, srv1558303981=20, srv591170030=75, srv1964256749=38, srv938367911=98, srv239680214=51, srv2101091745=46, srv82933296=86, srv280032993=56, srv1913015804=36, srv929135658=95, srv164853438=22, srv88573834=91, srv1870643237=33, srv528039816=69, srv1550884118=19, srv669744420=78, srv1432914081=15, srv1578680736=21, srv2005547657=41, srv1192246041=4, srv41184645=65, srv231171549=49, srv804515424=83, srv920970928=94, srv2003381805=40, srv1777308603=28, srv1452980935=16, srv104188442=0, srv1295097647=10, srv1926847855=37, srv807240750=85, srv634567242=77, srv205954963=45, srv847346856=88, srv780070327=82, srv2136308133=47, srv1092582754=2, srv1216679351=6, srv806394446=84, srv1740222354=25, srv262322822=55, srv537610337=71, srv1868205806=32, srv397471758=64, srv1889106797=35, srv1235553798=7, srv1055849666=1, srv1746708996=26, srv1738867983=24, srv613528067=76, srv233096350=50, srv755134548=79, srv1208196109=5, srv912415846=93, srv54112261=72, srv935775628=97, srv1352464290=12, srv1858723294=31, srv2013957265=42, srv253837984=52, srv1764388874=27, srv882409562=90, srv256549025=54, srv580566078=74, srv1374907824=13, srv1970947927=39, srv355393414=59, srv1833915426=30, srv930442462=96, srv1351181789=11, srv561225002=73, srv1388212027=14, srv901100843=92, srv349537603=58, srv776217962=81, srv2147202144=48, srv305726797=57, srv846394236=87, srv383780164=63, srv949450176=99, srv1540728694=18, srv1236580324=8, srv112335858=3, srv256516198=53, srv518424428=68, srv1522664035=17, srv2031786725=44, srv8680517=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:14,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:14,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-12 00:53:14,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-12 00:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-12 00:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-12 00:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-12 00:53:14,990 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005552187531265666); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007712569361139828); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005552187531265666); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:14,990 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-12 00:53:14,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.891826241537887E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005552187531265666); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007712569361139828); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005552187531265666); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-12 00:53:15,295 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 395 ms to try 5000 different iterations.  Found a solution that moves 259 regions; Going from a computed imbalance of 7.891826241537887E-4 to a new imbalance of 2.687010955452768E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.028777777777777777, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:15,296 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1764388874:81 , srv112335858:83 , srv1208196109:83 , srv1432914081:83 , srv1885146682:83 , srv397471758:83 , srv104188442:84 , srv1055849666:84 , srv1352464290:84 , srv1388212027:84 , srv1540728694:84 , srv1550884118:84 , srv1558303981:84 , srv1578680736:84 , srv164853438:84 , srv1657661728:84 , srv1738867983:84 , srv1740222354:84 , srv183225762:84 , srv1868205806:84 , srv1889106797:84 , srv1913015804:84 , srv2003381805:84 , srv2005547657:84 , srv2015456540:84 , srv2031786725:84 , srv205954963:84 , srv2101091745:84 , srv231171549:84 , srv256516198:84 , srv256549025:84 , srv262322822:84 , srv35616092:84 , srv41184645:84 , srv494445693:84 , srv634567242:84 , srv771333179:84 , srv780070327:84 , srv807240750:84 , srv846394236:84 , srv901100843:84 , srv929135658:84 , srv930442462:84 , srv1092582754:85 , srv1192246041:85 , srv1216679351:85 , srv1235553798:85 , srv1236580324:85 , srv1290216126:85 , srv1295097647:85 , srv1351181789:85 , srv1374907824:85 , srv1452980935:85 , srv1522664035:85 , srv1746708996:85 , srv1777308603:85 , srv1833915426:85 , srv1858723294:85 , srv1870643237:85 , srv1926847855:85 , srv1964256749:85 , srv1970947927:85 , srv2013957265:85 , srv2136308133:85 , srv2147202144:85 , srv233096350:85 , srv239680214:85 , srv253837984:85 , srv280032993:85 , srv305726797:85 , srv349537603:85 , srv355393414:85 , srv365082087:85 , srv381564646:85 , srv383780164:85 , srv493454760:85 , srv528039816:85 , srv535552269:85 , srv537610337:85 , srv54112261:85 , srv561225002:85 , srv580566078:85 , srv591170030:85 , srv613528067:85 , srv669744420:85 , srv755134548:85 , srv776217962:85 , srv804515424:85 , srv806394446:85 , srv82933296:85 , srv847346856:85 , srv8680517:85 , srv882409562:85 , srv88573834:85 , srv912415846:85 , srv920970928:85 , srv935775628:85 , srv938367911:85 , srv949450176:85 , srv518424428:636 }
2024-02-12 00:53:15,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:15,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1657661728=23, srv365082087=61, srv535552269=70, srv2015456540=43, srv771333179=80, srv381564646=62, srv494445693=67, srv183225762=29, srv1885146682=34, srv493454760=66, srv35616092=60, srv1290216126=9, srv1558303981=20, srv591170030=75, srv1964256749=38, srv938367911=98, srv239680214=51, srv2101091745=46, srv82933296=86, srv280032993=56, srv1913015804=36, srv929135658=95, srv164853438=22, srv88573834=91, srv1870643237=33, srv528039816=69, srv1550884118=19, srv669744420=78, srv1432914081=15, srv1578680736=21, srv2005547657=41, srv1192246041=4, srv41184645=65, srv231171549=49, srv804515424=83, srv920970928=94, srv2003381805=40, srv1777308603=28, srv1452980935=16, srv104188442=0, srv1295097647=10, srv1926847855=37, srv807240750=85, srv634567242=77, srv205954963=45, srv847346856=88, srv780070327=82, srv2136308133=47, srv1092582754=2, srv1216679351=6, srv806394446=84, srv1740222354=25, srv262322822=55, srv537610337=71, srv1868205806=32, srv397471758=64, srv1889106797=35, srv1235553798=7, srv1055849666=1, srv1746708996=26, srv1738867983=24, srv613528067=76, srv233096350=50, srv755134548=79, srv1208196109=5, srv912415846=93, srv54112261=72, srv935775628=97, srv1352464290=12, srv1858723294=31, srv2013957265=42, srv253837984=52, srv1764388874=27, srv882409562=90, srv256549025=54, srv580566078=74, srv1374907824=13, srv1970947927=39, srv355393414=59, srv1833915426=30, srv930442462=96, srv1351181789=11, srv561225002=73, srv1388212027=14, srv901100843=92, srv349537603=58, srv776217962=81, srv2147202144=48, srv305726797=57, srv846394236=87, srv383780164=63, srv949450176=99, srv1540728694=18, srv1236580324=8, srv112335858=3, srv256516198=53, srv518424428=68, srv1522664035=17, srv2031786725=44, srv8680517=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:15,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-12 00:53:15,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-12 00:53:15,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-12 00:53:15,313 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-12 00:53:15,314 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-12 00:53:15,315 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-12 00:53:15,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:15,409 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-02-12 00:53:15,409 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.505655279260177E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.158529131194229E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.158529131194229E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-02-12 00:53:15,605 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 282 ms to try 5000 different iterations.  Found a solution that moves 22 regions; Going from a computed imbalance of 2.505655279260177E-5 to a new imbalance of 7.2272865888482504E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0024444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:15,605 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1290216126:83 , srv755134548:83 , srv912415846:83 , srv104188442:84 , srv1055849666:84 , srv1092582754:84 , srv112335858:84 , srv1192246041:84 , srv1208196109:84 , srv1216679351:84 , srv1235553798:84 , srv1236580324:84 , srv1295097647:84 , srv1351181789:84 , srv1352464290:84 , srv1374907824:84 , srv1388212027:84 , srv1432914081:84 , srv1452980935:84 , srv1522664035:84 , srv1540728694:84 , srv1550884118:84 , srv1558303981:84 , srv1578680736:84 , srv1657661728:84 , srv1738867983:84 , srv1740222354:84 , srv1746708996:84 , srv1764388874:84 , srv1777308603:84 , srv183225762:84 , srv1833915426:84 , srv1858723294:84 , srv1868205806:84 , srv1870643237:84 , srv1885146682:84 , srv1913015804:84 , srv1926847855:84 , srv1964256749:84 , srv1970947927:84 , srv2005547657:84 , srv2013957265:84 , srv2015456540:84 , srv2031786725:84 , srv205954963:84 , srv2101091745:84 , srv2136308133:84 , srv2147202144:84 , srv231171549:84 , srv233096350:84 , srv239680214:84 , srv253837984:84 , srv256516198:84 , srv256549025:84 , srv262322822:84 , srv280032993:84 , srv305726797:84 , srv349537603:84 , srv355393414:84 , srv35616092:84 , srv365082087:84 , srv381564646:84 , srv383780164:84 , srv397471758:84 , srv41184645:84 , srv493454760:84 , srv494445693:84 , srv528039816:84 , srv535552269:84 , srv537610337:84 , srv54112261:84 , srv561225002:84 , srv580566078:84 , srv591170030:84 , srv613528067:84 , srv634567242:84 , srv669744420:84 , srv776217962:84 , srv780070327:84 , srv804515424:84 , srv806394446:84 , srv807240750:84 , srv82933296:84 , srv846394236:84 , srv847346856:84 , srv8680517:84 , srv882409562:84 , srv88573834:84 , srv901100843:84 , srv920970928:84 , srv929135658:84 , srv930442462:84 , srv935775628:84 , srv938367911:84 , srv949450176:84 , srv164853438:85 , srv1889106797:85 , srv2003381805:85 , srv771333179:85 , srv518424428:683 }
2024-02-12 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1657661728=23, srv365082087=61, srv535552269=70, srv2015456540=43, srv771333179=80, srv381564646=62, srv494445693=67, srv183225762=29, srv1885146682=34, srv493454760=66, srv35616092=60, srv1290216126=9, srv1558303981=20, srv591170030=75, srv1964256749=38, srv938367911=98, srv239680214=51, srv2101091745=46, srv82933296=86, srv280032993=56, srv1913015804=36, srv929135658=95, srv164853438=22, srv88573834=91, srv1870643237=33, srv528039816=69, srv1550884118=19, srv669744420=78, srv1432914081=15, srv1578680736=21, srv2005547657=41, srv1192246041=4, srv41184645=65, srv231171549=49, srv804515424=83, srv920970928=94, srv2003381805=40, srv1777308603=28, srv1452980935=16, srv104188442=0, srv1295097647=10, srv1926847855=37, srv807240750=85, srv634567242=77, srv205954963=45, srv847346856=88, srv780070327=82, srv2136308133=47, srv1092582754=2, srv1216679351=6, srv806394446=84, srv1740222354=25, srv262322822=55, srv537610337=71, srv1868205806=32, srv397471758=64, srv1889106797=35, srv1235553798=7, srv1055849666=1, srv1746708996=26, srv1738867983=24, srv613528067=76, srv233096350=50, srv755134548=79, srv1208196109=5, srv912415846=93, srv54112261=72, srv935775628=97, srv1352464290=12, srv1858723294=31, srv2013957265=42, srv253837984=52, srv1764388874=27, srv882409562=90, srv256549025=54, srv580566078=74, srv1374907824=13, srv1970947927=39, srv355393414=59, srv1833915426=30, srv930442462=96, srv1351181789=11, srv561225002=73, srv1388212027=14, srv901100843=92, srv349537603=58, srv776217962=81, srv2147202144=48, srv305726797=57, srv846394236=87, srv383780164=63, srv949450176=99, srv1540728694=18, srv1236580324=8, srv112335858=3, srv256516198=53, srv518424428=68, srv1522664035=17, srv2031786725=44, srv8680517=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:15,626 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-02-12 00:53:15,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-02-12 00:53:15,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-02-12 00:53:15,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-02-12 00:53:15,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-02-12 00:53:15,727 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:15,727 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1290216126:83 , srv755134548:83 , srv912415846:83 , srv104188442:84 , srv1055849666:84 , srv1092582754:84 , srv112335858:84 , srv1192246041:84 , srv1208196109:84 , srv1216679351:84 , srv1235553798:84 , srv1236580324:84 , srv1295097647:84 , srv1351181789:84 , srv1352464290:84 , srv1374907824:84 , srv1388212027:84 , srv1432914081:84 , srv1452980935:84 , srv1522664035:84 , srv1540728694:84 , srv1550884118:84 , srv1558303981:84 , srv1578680736:84 , srv1657661728:84 , srv1738867983:84 , srv1740222354:84 , srv1746708996:84 , srv1764388874:84 , srv1777308603:84 , srv183225762:84 , srv1833915426:84 , srv1858723294:84 , srv1868205806:84 , srv1870643237:84 , srv1885146682:84 , srv1913015804:84 , srv1926847855:84 , srv1964256749:84 , srv1970947927:84 , srv2005547657:84 , srv2013957265:84 , srv2015456540:84 , srv2031786725:84 , srv205954963:84 , srv2101091745:84 , srv2136308133:84 , srv2147202144:84 , srv231171549:84 , srv233096350:84 , srv239680214:84 , srv253837984:84 , srv256516198:84 , srv256549025:84 , srv262322822:84 , srv280032993:84 , srv305726797:84 , srv349537603:84 , srv355393414:84 , srv35616092:84 , srv365082087:84 , srv381564646:84 , srv383780164:84 , srv397471758:84 , srv41184645:84 , srv493454760:84 , srv494445693:84 , srv528039816:84 , srv535552269:84 , srv537610337:84 , srv54112261:84 , srv561225002:84 , srv580566078:84 , srv591170030:84 , srv613528067:84 , srv634567242:84 , srv669744420:84 , srv776217962:84 , srv780070327:84 , srv804515424:84 , srv806394446:84 , srv807240750:84 , srv82933296:84 , srv846394236:84 , srv847346856:84 , srv8680517:84 , srv882409562:84 , srv88573834:84 , srv901100843:84 , srv920970928:84 , srv929135658:84 , srv930442462:84 , srv935775628:84 , srv938367911:84 , srv949450176:84 , srv164853438:85 , srv1889106797:85 , srv2003381805:85 , srv771333179:85 , srv518424428:683 }
2024-02-12 00:53:15,751 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=343 (was 286) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=2560 (was 3311)
2024-02-12 00:53:15,758 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=343, ProcessCount=170, AvailableMemoryMB=2559
2024-02-12 00:53:15,759 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-02-12 00:53:15,759 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-02-12 00:53:15,770 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1014719314:84 , srv1020657902:84 , srv1045971989:84 , srv1098029811:84 , srv1169406886:84 , srv1219878941:84 , srv1241999032:84 , srv1418143571:84 , srv1467389903:84 , srv1587381707:84 , srv1746884397:84 , srv1916746547:84 , srv1945190709:84 , srv1993333786:84 , srv2003810557:84 , srv2046181217:84 , srv2066038917:84 , srv2128319053:84 , srv2144918203:84 , srv319964705:84 , srv361541512:84 , srv363830709:84 , srv494590702:84 , srv64117541:84 , srv760269653:84 , srv856062030:84 , srv866911953:84 , srv882708147:84 , srv899750896:84 , srv1449654004:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-02-12 00:53:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:53:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2046181217=16, srv1916746547=12, srv1169406886=4, srv1045971989=2, srv494590702=23, srv1746884397=11, srv363830709=22, srv64117541=24, srv1014719314=0, srv760269653=25, srv1449654004=8, srv1993333786=14, srv1020657902=1, srv2003810557=15, srv1219878941=5, srv1467389903=9, srv866911953=27, srv1587381707=10, srv361541512=21, srv2144918203=19, srv2066038917=17, srv2128319053=18, srv1945190709=13, srv1241999032=6, srv882708147=28, srv319964705=20, srv856062030=26, srv899750896=29, srv1098029811=3, srv1418143571=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:53:15,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:53:15,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:53:15,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:53:15,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:53:15,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:53:15,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:53:15,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-12 00:53:15,810 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1445835817556942, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:53:15,810 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912648621409303, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1445835817556942, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-02-12 00:55:15,790 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 7314605 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912648621409303 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:55:15,795 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1014719314:90 , srv1020657902:90 , srv1045971989:90 , srv1098029811:90 , srv1169406886:90 , srv1219878941:90 , srv1241999032:90 , srv1418143571:90 , srv1449654004:90 , srv1467389903:90 , srv1587381707:90 , srv1746884397:90 , srv1916746547:90 , srv1945190709:90 , srv1993333786:90 , srv2003810557:90 , srv2046181217:90 , srv2066038917:90 , srv2128319053:90 , srv2144918203:90 , srv319964705:90 , srv361541512:90 , srv363830709:90 , srv494590702:90 , srv64117541:90 , srv760269653:90 , srv856062030:90 , srv866911953:90 , srv882708147:90 , srv899750896:90 }
2024-02-12 00:55:15,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-02-12 00:55:15,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2046181217=16, srv1916746547=12, srv1169406886=4, srv1045971989=2, srv494590702=23, srv1746884397=11, srv363830709=22, srv64117541=24, srv1014719314=0, srv760269653=25, srv1449654004=8, srv1993333786=14, srv1020657902=1, srv2003810557=15, srv1219878941=5, srv1467389903=9, srv866911953=27, srv1587381707=10, srv361541512=21, srv2144918203=19, srv2066038917=17, srv2128319053=18, srv1945190709=13, srv1241999032=6, srv882708147=28, srv319964705=20, srv856062030=26, srv899750896=29, srv1098029811=3, srv1418143571=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-02-12 00:55:15,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-02-12 00:55:15,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-02-12 00:55:15,812 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-02-12 00:55:15,812 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1014719314:90 , srv1020657902:90 , srv1045971989:90 , srv1098029811:90 , srv1169406886:90 , srv1219878941:90 , srv1241999032:90 , srv1418143571:90 , srv1449654004:90 , srv1467389903:90 , srv1587381707:90 , srv1746884397:90 , srv1916746547:90 , srv1945190709:90 , srv1993333786:90 , srv2003810557:90 , srv2046181217:90 , srv2066038917:90 , srv2128319053:90 , srv2144918203:90 , srv319964705:90 , srv361541512:90 , srv363830709:90 , srv494590702:90 , srv64117541:90 , srv760269653:90 , srv856062030:90 , srv866911953:90 , srv882708147:90 , srv899750896:90 }
2024-02-12 00:55:15,823 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=166 (was 343), ProcessCount=171 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=2139 (was 2559)