2023-10-08 11:57:01,423 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-08 11:57:01,664 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-08 11:57:01,749 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-08 11:57:01,800 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-08 11:57:01,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-08 11:57:01,817 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=227, ProcessCount=168, AvailableMemoryMB=3917
2023-10-08 11:57:01,819 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-08 11:57:01,819 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-08 11:57:01,933 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1005233533:84 , srv1101720880:84 , srv1107503175:84 , srv1135625904:84 , srv1151807906:84 , srv115428851:84 , srv1171260116:84 , srv1234847252:84 , srv1246285007:84 , srv1323311448:84 , srv1330496928:84 , srv1376034173:84 , srv1390536382:84 , srv1394793626:84 , srv1397938204:84 , srv1405837199:84 , srv1418415944:84 , srv1421022123:84 , srv1429762742:84 , srv1455591196:84 , srv1502611692:84 , srv1514431040:84 , srv1520843572:84 , srv1565683344:84 , srv1582851939:84 , srv1588436854:84 , srv1593026093:84 , srv1595102154:84 , srv1599295152:84 , srv1607171848:84 , srv1610221893:84 , srv1612716927:84 , srv1621277365:84 , srv1683987373:84 , srv1687833178:84 , srv1697026619:84 , srv1698753885:84 , srv1725953848:84 , srv1746859422:84 , srv1751601412:84 , srv1753759082:84 , srv175807952:84 , srv1779004378:84 , srv1800094483:84 , srv1801542256:84 , srv1817485837:84 , srv1832909915:84 , srv1841176509:84 , srv185790356:84 , srv1863595173:84 , srv1864447003:84 , srv187057400:84 , srv1875719440:84 , srv1910519708:84 , srv1932690537:84 , srv1963537254:84 , srv1979271972:84 , srv200700916:84 , srv2013124454:84 , srv202338056:84 , srv202985091:84 , srv2056679767:84 , srv2068502106:84 , srv2073486758:84 , srv2111132818:84 , srv217829243:84 , srv225245512:84 , srv248910583:84 , srv257264149:84 , srv324505696:84 , srv339098536:84 , srv37073469:84 , srv410074558:84 , srv465690368:84 , srv471724878:84 , srv484787487:84 , srv502005008:84 , srv519811277:84 , srv592038463:84 , srv608557487:84 , srv62437199:84 , srv670205242:84 , srv712624724:84 , srv722027430:84 , srv745726284:84 , srv784201645:84 , srv788826737:84 , srv790003351:84 , srv796489474:84 , srv802546454:84 , srv804476407:84 , srv817369744:84 , srv914026760:84 , srv917051432:84 , srv932625031:84 , srv951937966:84 , srv964140511:84 , srv982949874:84 , srv999637667:84 , srv1137072971:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-08 11:57:01,940 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:01,946 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv712624724=83, srv1514431040=22, srv1863595173=50, srv1698753885=37, srv1932690537=55, srv465690368=74, srv1593026093=27, srv257264149=69, srv914026760=93, srv1683987373=34, srv1246285007=9, srv2111132818=65, srv1405837199=16, srv592038463=79, srv1753759082=41, srv2013124454=59, srv1817485837=46, srv1779004378=43, srv1725953848=38, srv1418415944=17, srv1421022123=18, srv917051432=94, srv62437199=81, srv804476407=91, srv1137072971=4, srv982949874=98, srv1376034173=12, srv1612716927=32, srv1979271972=57, srv2073486758=64, srv202985091=61, srv1330496928=11, srv1397938204=15, srv1234847252=8, srv1687833178=35, srv484787487=76, srv951937966=96, srv1565683344=24, srv1832909915=47, srv202338056=60, srv999637667=99, srv1746859422=39, srv248910583=68, srv1101720880=1, srv1875719440=53, srv964140511=97, srv324505696=70, srv1607171848=30, srv1751601412=40, srv185790356=49, srv1171260116=7, srv1582851939=25, srv784201645=86, srv1801542256=45, srv37073469=72, srv1455591196=20, srv802546454=90, srv1135625904=3, srv1800094483=44, srv1107503175=2, srv1520843572=23, srv608557487=80, srv175807952=42, srv1323311448=10, srv115428851=6, srv1502611692=21, srv1910519708=54, srv1610221893=31, srv722027430=84, srv200700916=58, srv932625031=95, srv745726284=85, srv1841176509=48, srv471724878=75, srv1429762742=19, srv217829243=66, srv788826737=87, srv187057400=52, srv339098536=71, srv2068502106=63, srv1151807906=5, srv410074558=73, srv790003351=88, srv1963537254=56, srv1697026619=36, srv670205242=82, srv519811277=78, srv1621277365=33, srv1588436854=26, srv1864447003=51, srv225245512=67, srv817369744=92, srv1599295152=29, srv796489474=89, srv1394793626=14, srv2056679767=62, srv1595102154=28, srv1390536382=13, srv502005008=77, srv1005233533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:02,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:02,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:02,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:02,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-08 11:57:02,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-08 11:57:02,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-08 11:57:02,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-08 11:57:02,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-08 11:57:02,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-08 11:57:02,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-08 11:57:02,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-08 11:57:02,156 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:02,157 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-08 11:57:02,157 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-08 11:57:02,391 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 340 ms to try 5000 different iterations.  Found a solution that moves 4711 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17892058418051543. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0394101667346147, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055093024076628506, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5234444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0394101667346147, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2405, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:02,403 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv592038463:81 , srv1687833178:82 , srv1817485837:82 , srv2068502106:82 , srv217829243:82 , srv257264149:82 , srv932625031:82 , srv1246285007:83 , srv1520843572:83 , srv1725953848:83 , srv1841176509:83 , srv1864447003:83 , srv519811277:83 , srv1101720880:84 , srv1135625904:84 , srv1323311448:84 , srv1588436854:84 , srv1753759082:84 , srv1779004378:84 , srv1800094483:84 , srv185790356:84 , srv796489474:84 , srv802546454:84 , srv804476407:84 , srv1234847252:85 , srv1330496928:85 , srv1455591196:85 , srv1610221893:85 , srv1697026619:85 , srv1698753885:85 , srv1963537254:85 , srv2111132818:85 , srv225245512:85 , srv339098536:85 , srv410074558:85 , srv784201645:85 , srv914026760:85 , srv951937966:85 , srv982949874:85 , srv1151807906:86 , srv1171260116:86 , srv1376034173:86 , srv1397938204:86 , srv1502611692:86 , srv1599295152:86 , srv1607171848:86 , srv1683987373:86 , srv1751601412:86 , srv1863595173:86 , srv1875719440:86 , srv324505696:86 , srv471724878:86 , srv745726284:86 , srv788826737:86 , srv964140511:86 , srv999637667:86 , srv1107503175:87 , srv115428851:87 , srv1390536382:87 , srv1421022123:87 , srv1429762742:87 , srv1612716927:87 , srv1621277365:87 , srv1832909915:87 , srv187057400:87 , srv2013124454:87 , srv2073486758:87 , srv37073469:87 , srv502005008:87 , srv917051432:87 , srv1418415944:88 , srv175807952:88 , srv1932690537:88 , srv200700916:88 , srv202338056:88 , srv248910583:88 , srv62437199:88 , srv670205242:88 , srv790003351:88 , srv1005233533:89 , srv1405837199:89 , srv202985091:89 , srv2056679767:89 , srv1582851939:90 , srv1801542256:90 , srv1979271972:90 , srv465690368:90 , srv484787487:90 , srv712624724:90 , srv1394793626:91 , srv1514431040:91 , srv1565683344:91 , srv1595102154:91 , srv1746859422:91 , srv817369744:91 , srv1593026093:92 , srv722027430:93 , srv1910519708:95 , srv608557487:96 , srv1137072971:440 }
2023-10-08 11:57:02,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:02,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv712624724=83, srv1514431040=22, srv1863595173=50, srv1698753885=37, srv1932690537=55, srv465690368=74, srv1593026093=27, srv257264149=69, srv914026760=93, srv1683987373=34, srv1246285007=9, srv2111132818=65, srv1405837199=16, srv592038463=79, srv1753759082=41, srv2013124454=59, srv1817485837=46, srv1779004378=43, srv1725953848=38, srv1418415944=17, srv1421022123=18, srv917051432=94, srv62437199=81, srv804476407=91, srv1137072971=4, srv982949874=98, srv1376034173=12, srv1612716927=32, srv1979271972=57, srv2073486758=64, srv202985091=61, srv1330496928=11, srv1397938204=15, srv1234847252=8, srv1687833178=35, srv484787487=76, srv951937966=96, srv1565683344=24, srv1832909915=47, srv202338056=60, srv999637667=99, srv1746859422=39, srv248910583=68, srv1101720880=1, srv1875719440=53, srv964140511=97, srv324505696=70, srv1607171848=30, srv1751601412=40, srv185790356=49, srv1171260116=7, srv1582851939=25, srv784201645=86, srv1801542256=45, srv37073469=72, srv1455591196=20, srv802546454=90, srv1135625904=3, srv1800094483=44, srv1107503175=2, srv1520843572=23, srv608557487=80, srv175807952=42, srv1323311448=10, srv115428851=6, srv1502611692=21, srv1910519708=54, srv1610221893=31, srv722027430=84, srv200700916=58, srv932625031=95, srv745726284=85, srv1841176509=48, srv471724878=75, srv1429762742=19, srv217829243=66, srv788826737=87, srv187057400=52, srv339098536=71, srv2068502106=63, srv1151807906=5, srv410074558=73, srv790003351=88, srv1963537254=56, srv1697026619=36, srv670205242=82, srv519811277=78, srv1621277365=33, srv1588436854=26, srv1864447003=51, srv225245512=67, srv817369744=92, srv1599295152=29, srv796489474=89, srv1394793626=14, srv2056679767=62, srv1595102154=28, srv1390536382=13, srv502005008=77, srv1005233533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:02,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:02,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:02,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:02,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:02,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-08 11:57:02,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-08 11:57:02,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-08 11:57:02,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-08 11:57:02,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-08 11:57:02,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-08 11:57:02,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0394101667346147, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055093024076628506, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0394101667346147, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2405, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:02,518 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-08 11:57:02,518 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1788875970534674, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0394101667346147, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055093024076628506, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0394101667346147, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17416666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2405, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-08 11:57:02,704 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 267 ms to try 5000 different iterations.  Found a solution that moves 2659 regions; Going from a computed imbalance of 0.1788875970534674 to a new imbalance of 0.014617311180610259. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01935871754869923); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030777431351293984, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29544444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01935871754869923); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04541666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:02,712 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1910519708:77 , srv608557487:78 , srv1394793626:80 , srv1593026093:80 , srv1746859422:80 , srv1514431040:81 , srv1582851939:81 , srv1595102154:81 , srv817369744:81 , srv1390536382:82 , srv1565683344:82 , srv1979271972:82 , srv484787487:82 , srv722027430:82 , srv1005233533:83 , srv1418415944:83 , srv1683987373:83 , srv1801542256:83 , srv202338056:83 , srv465690368:83 , srv62437199:83 , srv712624724:83 , srv1429762742:84 , srv1612716927:84 , srv200700916:84 , srv248910583:84 , srv670205242:84 , srv790003351:84 , srv1151807906:85 , srv115428851:85 , srv1234847252:85 , srv1376034173:85 , srv1397938204:85 , srv1405837199:85 , srv175807952:85 , srv187057400:85 , srv1932690537:85 , srv202985091:85 , srv2056679767:85 , srv37073469:85 , srv745726284:85 , srv1107503175:86 , srv1171260116:86 , srv1421022123:86 , srv1455591196:86 , srv1599295152:86 , srv1607171848:86 , srv1621277365:86 , srv1698753885:86 , srv1751601412:86 , srv1832909915:86 , srv1863595173:86 , srv1875719440:86 , srv2013124454:86 , srv2073486758:86 , srv225245512:86 , srv324505696:86 , srv339098536:86 , srv471724878:86 , srv502005008:86 , srv951937966:86 , srv964140511:86 , srv999637667:86 , srv1135625904:87 , srv1323311448:87 , srv1330496928:87 , srv1610221893:87 , srv1697026619:87 , srv1753759082:87 , srv1800094483:87 , srv1963537254:87 , srv2111132818:87 , srv410074558:87 , srv519811277:87 , srv804476407:87 , srv914026760:87 , srv917051432:87 , srv982949874:87 , srv784201645:88 , srv788826737:88 , srv802546454:88 , srv1779004378:89 , srv1841176509:89 , srv185790356:89 , srv796489474:89 , srv1101720880:90 , srv1725953848:90 , srv1864447003:90 , srv217829243:90 , srv257264149:90 , srv1246285007:91 , srv1502611692:91 , srv1687833178:91 , srv592038463:91 , srv1520843572:92 , srv1588436854:92 , srv2068502106:92 , srv932625031:92 , srv1817485837:93 , srv1137072971:506 }
2023-10-08 11:57:02,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:02,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv712624724=83, srv1514431040=22, srv1863595173=50, srv1698753885=37, srv1932690537=55, srv465690368=74, srv1593026093=27, srv257264149=69, srv914026760=93, srv1683987373=34, srv1246285007=9, srv2111132818=65, srv1405837199=16, srv592038463=79, srv1753759082=41, srv2013124454=59, srv1817485837=46, srv1779004378=43, srv1725953848=38, srv1418415944=17, srv1421022123=18, srv917051432=94, srv62437199=81, srv804476407=91, srv1137072971=4, srv982949874=98, srv1376034173=12, srv1612716927=32, srv1979271972=57, srv2073486758=64, srv202985091=61, srv1330496928=11, srv1397938204=15, srv1234847252=8, srv1687833178=35, srv484787487=76, srv951937966=96, srv1565683344=24, srv1832909915=47, srv202338056=60, srv999637667=99, srv1746859422=39, srv248910583=68, srv1101720880=1, srv1875719440=53, srv964140511=97, srv324505696=70, srv1607171848=30, srv1751601412=40, srv185790356=49, srv1171260116=7, srv1582851939=25, srv784201645=86, srv1801542256=45, srv37073469=72, srv1455591196=20, srv802546454=90, srv1135625904=3, srv1800094483=44, srv1107503175=2, srv1520843572=23, srv608557487=80, srv175807952=42, srv1323311448=10, srv115428851=6, srv1502611692=21, srv1910519708=54, srv1610221893=31, srv722027430=84, srv200700916=58, srv932625031=95, srv745726284=85, srv1841176509=48, srv471724878=75, srv1429762742=19, srv217829243=66, srv788826737=87, srv187057400=52, srv339098536=71, srv2068502106=63, srv1151807906=5, srv410074558=73, srv790003351=88, srv1963537254=56, srv1697026619=36, srv670205242=82, srv519811277=78, srv1621277365=33, srv1588436854=26, srv1864447003=51, srv225245512=67, srv817369744=92, srv1599295152=29, srv796489474=89, srv1394793626=14, srv2056679767=62, srv1595102154=28, srv1390536382=13, srv502005008=77, srv1005233533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:02,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:02,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:02,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-08 11:57:02,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-08 11:57:02,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-08 11:57:02,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-08 11:57:02,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-08 11:57:02,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-08 11:57:02,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-08 11:57:02,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-08 11:57:02,844 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01935871754869923); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030777431351293984, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01935871754869923); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04541666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:02,844 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-08 11:57:02,845 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014598692464664465, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01935871754869923); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.030777431351293984, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.01935871754869923); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011416666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04541666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-08 11:57:03,010 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 247 ms to try 5000 different iterations.  Found a solution that moves 992 regions; Going from a computed imbalance of 0.014598692464664465 to a new imbalance of 6.649055303096884E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005883687854222971); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008206099398622182); MoveCostFunction : (multiplier=7.0, imbalance=0.11022222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005883687854222971); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,011 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1502611692:81 , srv1588436854:81 , srv1817485837:82 , srv1101720880:83 , srv1246285007:83 , srv1520843572:83 , srv202985091:83 , srv2056679767:83 , srv722027430:83 , srv1107503175:84 , srv1405837199:84 , srv1514431040:84 , srv1607171848:84 , srv1621277365:84 , srv1779004378:84 , srv1832909915:84 , srv1932690537:84 , srv2068502106:84 , srv2073486758:84 , srv465690368:84 , srv608557487:84 , srv712624724:84 , srv788826737:84 , srv796489474:84 , srv917051432:84 , srv932625031:84 , srv1171260116:85 , srv1421022123:85 , srv1565683344:85 , srv1595102154:85 , srv1599295152:85 , srv1610221893:85 , srv1687833178:85 , srv1697026619:85 , srv1725953848:85 , srv1751601412:85 , srv175807952:85 , srv1801542256:85 , srv185790356:85 , srv1863595173:85 , srv1864447003:85 , srv187057400:85 , srv1963537254:85 , srv200700916:85 , srv2013124454:85 , srv217829243:85 , srv324505696:85 , srv37073469:85 , srv410074558:85 , srv484787487:85 , srv502005008:85 , srv784201645:85 , srv802546454:85 , srv804476407:85 , srv964140511:85 , srv1005233533:86 , srv1135625904:86 , srv1151807906:86 , srv115428851:86 , srv1234847252:86 , srv1376034173:86 , srv1394793626:86 , srv1397938204:86 , srv1455591196:86 , srv1593026093:86 , srv1612716927:86 , srv1698753885:86 , srv1800094483:86 , srv1841176509:86 , srv1875719440:86 , srv1979271972:86 , srv2111132818:86 , srv225245512:86 , srv248910583:86 , srv339098536:86 , srv471724878:86 , srv519811277:86 , srv592038463:86 , srv670205242:86 , srv745726284:86 , srv790003351:86 , srv817369744:86 , srv914026760:86 , srv951937966:86 , srv982949874:86 , srv999637667:86 , srv1323311448:87 , srv1418415944:87 , srv1429762742:87 , srv1582851939:87 , srv1746859422:87 , srv1753759082:87 , srv1910519708:87 , srv202338056:87 , srv257264149:87 , srv62437199:87 , srv1330496928:88 , srv1683987373:88 , srv1390536382:89 , srv1137072971:564 }
2023-10-08 11:57:03,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:03,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv712624724=83, srv1514431040=22, srv1863595173=50, srv1698753885=37, srv1932690537=55, srv465690368=74, srv1593026093=27, srv257264149=69, srv914026760=93, srv1683987373=34, srv1246285007=9, srv2111132818=65, srv1405837199=16, srv592038463=79, srv1753759082=41, srv2013124454=59, srv1817485837=46, srv1779004378=43, srv1725953848=38, srv1418415944=17, srv1421022123=18, srv917051432=94, srv62437199=81, srv804476407=91, srv1137072971=4, srv982949874=98, srv1376034173=12, srv1612716927=32, srv1979271972=57, srv2073486758=64, srv202985091=61, srv1330496928=11, srv1397938204=15, srv1234847252=8, srv1687833178=35, srv484787487=76, srv951937966=96, srv1565683344=24, srv1832909915=47, srv202338056=60, srv999637667=99, srv1746859422=39, srv248910583=68, srv1101720880=1, srv1875719440=53, srv964140511=97, srv324505696=70, srv1607171848=30, srv1751601412=40, srv185790356=49, srv1171260116=7, srv1582851939=25, srv784201645=86, srv1801542256=45, srv37073469=72, srv1455591196=20, srv802546454=90, srv1135625904=3, srv1800094483=44, srv1107503175=2, srv1520843572=23, srv608557487=80, srv175807952=42, srv1323311448=10, srv115428851=6, srv1502611692=21, srv1910519708=54, srv1610221893=31, srv722027430=84, srv200700916=58, srv932625031=95, srv745726284=85, srv1841176509=48, srv471724878=75, srv1429762742=19, srv217829243=66, srv788826737=87, srv187057400=52, srv339098536=71, srv2068502106=63, srv1151807906=5, srv410074558=73, srv790003351=88, srv1963537254=56, srv1697026619=36, srv670205242=82, srv519811277=78, srv1621277365=33, srv1588436854=26, srv1864447003=51, srv225245512=67, srv817369744=92, srv1599295152=29, srv796489474=89, srv1394793626=14, srv2056679767=62, srv1595102154=28, srv1390536382=13, srv502005008=77, srv1005233533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:03,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:03,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-08 11:57:03,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-08 11:57:03,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-08 11:57:03,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-08 11:57:03,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-08 11:57:03,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-08 11:57:03,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-08 11:57:03,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-08 11:57:03,122 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005883687854222971); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008206099398622182); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005883687854222971); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,122 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-08 11:57:03,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.579593978470225E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005883687854222971); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008206099398622182); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005883687854222971); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-08 11:57:03,339 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 298 ms to try 5000 different iterations.  Found a solution that moves 261 regions; Going from a computed imbalance of 6.579593978470225E-4 to a new imbalance of 2.6955134383429384E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.029, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,340 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1330496928:82 , srv1910519708:83 , srv1005233533:84 , srv115428851:84 , srv1323311448:84 , srv1390536382:84 , srv1394793626:84 , srv1405837199:84 , srv1418415944:84 , srv1421022123:84 , srv1429762742:84 , srv1502611692:84 , srv1520843572:84 , srv1565683344:84 , srv1582851939:84 , srv1593026093:84 , srv1687833178:84 , srv1725953848:84 , srv1746859422:84 , srv1751601412:84 , srv1753759082:84 , srv175807952:84 , srv1801542256:84 , srv1841176509:84 , srv185790356:84 , srv1864447003:84 , srv1875719440:84 , srv1963537254:84 , srv1979271972:84 , srv202338056:84 , srv2068502106:84 , srv2111132818:84 , srv248910583:84 , srv257264149:84 , srv484787487:84 , srv502005008:84 , srv608557487:84 , srv62437199:84 , srv670205242:84 , srv722027430:84 , srv745726284:84 , srv784201645:84 , srv788826737:84 , srv817369744:84 , srv914026760:84 , srv917051432:84 , srv932625031:84 , srv982949874:84 , srv999637667:84 , srv1101720880:85 , srv1107503175:85 , srv1135625904:85 , srv1151807906:85 , srv1171260116:85 , srv1246285007:85 , srv1376034173:85 , srv1397938204:85 , srv1455591196:85 , srv1514431040:85 , srv1588436854:85 , srv1595102154:85 , srv1599295152:85 , srv1610221893:85 , srv1612716927:85 , srv1621277365:85 , srv1683987373:85 , srv1697026619:85 , srv1698753885:85 , srv1779004378:85 , srv1800094483:85 , srv1817485837:85 , srv1832909915:85 , srv1863595173:85 , srv187057400:85 , srv1932690537:85 , srv200700916:85 , srv2013124454:85 , srv202985091:85 , srv2056679767:85 , srv2073486758:85 , srv217829243:85 , srv225245512:85 , srv324505696:85 , srv37073469:85 , srv410074558:85 , srv465690368:85 , srv471724878:85 , srv592038463:85 , srv712624724:85 , srv790003351:85 , srv796489474:85 , srv802546454:85 , srv951937966:85 , srv964140511:85 , srv1234847252:86 , srv1607171848:86 , srv339098536:86 , srv519811277:86 , srv804476407:86 , srv1137072971:632 }
2023-10-08 11:57:03,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:03,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv712624724=83, srv1514431040=22, srv1863595173=50, srv1698753885=37, srv1932690537=55, srv465690368=74, srv1593026093=27, srv257264149=69, srv914026760=93, srv1683987373=34, srv1246285007=9, srv2111132818=65, srv1405837199=16, srv592038463=79, srv1753759082=41, srv2013124454=59, srv1817485837=46, srv1779004378=43, srv1725953848=38, srv1418415944=17, srv1421022123=18, srv917051432=94, srv62437199=81, srv804476407=91, srv1137072971=4, srv982949874=98, srv1376034173=12, srv1612716927=32, srv1979271972=57, srv2073486758=64, srv202985091=61, srv1330496928=11, srv1397938204=15, srv1234847252=8, srv1687833178=35, srv484787487=76, srv951937966=96, srv1565683344=24, srv1832909915=47, srv202338056=60, srv999637667=99, srv1746859422=39, srv248910583=68, srv1101720880=1, srv1875719440=53, srv964140511=97, srv324505696=70, srv1607171848=30, srv1751601412=40, srv185790356=49, srv1171260116=7, srv1582851939=25, srv784201645=86, srv1801542256=45, srv37073469=72, srv1455591196=20, srv802546454=90, srv1135625904=3, srv1800094483=44, srv1107503175=2, srv1520843572=23, srv608557487=80, srv175807952=42, srv1323311448=10, srv115428851=6, srv1502611692=21, srv1910519708=54, srv1610221893=31, srv722027430=84, srv200700916=58, srv932625031=95, srv745726284=85, srv1841176509=48, srv471724878=75, srv1429762742=19, srv217829243=66, srv788826737=87, srv187057400=52, srv339098536=71, srv2068502106=63, srv1151807906=5, srv410074558=73, srv790003351=88, srv1963537254=56, srv1697026619=36, srv670205242=82, srv519811277=78, srv1621277365=33, srv1588436854=26, srv1864447003=51, srv225245512=67, srv817369744=92, srv1599295152=29, srv796489474=89, srv1394793626=14, srv2056679767=62, srv1595102154=28, srv1390536382=13, srv502005008=77, srv1005233533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:03,357 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:03,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:03,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-08 11:57:03,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-08 11:57:03,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-08 11:57:03,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-08 11:57:03,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-08 11:57:03,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-08 11:57:03,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-08 11:57:03,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-08 11:57:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-08 11:57:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-08 11:57:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-08 11:57:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-08 11:57:03,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-08 11:57:03,443 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,443 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-08 11:57:03,444 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.5127573322183584E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-08 11:57:03,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 215 ms to try 5000 different iterations.  Found a solution that moves 22 regions; Going from a computed imbalance of 2.5127573322183584E-5 to a new imbalance of 6.898116346350751E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0024444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,590 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2013124454:83 , srv257264149:83 , srv339098536:83 , srv471724878:83 , srv592038463:83 , srv790003351:83 , srv1005233533:84 , srv1101720880:84 , srv1107503175:84 , srv1135625904:84 , srv1151807906:84 , srv115428851:84 , srv1171260116:84 , srv1234847252:84 , srv1246285007:84 , srv1323311448:84 , srv1330496928:84 , srv1376034173:84 , srv1390536382:84 , srv1397938204:84 , srv1405837199:84 , srv1418415944:84 , srv1421022123:84 , srv1429762742:84 , srv1455591196:84 , srv1502611692:84 , srv1520843572:84 , srv1565683344:84 , srv1582851939:84 , srv1588436854:84 , srv1593026093:84 , srv1595102154:84 , srv1599295152:84 , srv1607171848:84 , srv1610221893:84 , srv1612716927:84 , srv1621277365:84 , srv1683987373:84 , srv1687833178:84 , srv1697026619:84 , srv1698753885:84 , srv1725953848:84 , srv1746859422:84 , srv1753759082:84 , srv175807952:84 , srv1779004378:84 , srv1800094483:84 , srv1801542256:84 , srv1817485837:84 , srv1832909915:84 , srv1841176509:84 , srv185790356:84 , srv1863595173:84 , srv1864447003:84 , srv187057400:84 , srv1875719440:84 , srv1910519708:84 , srv1932690537:84 , srv1979271972:84 , srv200700916:84 , srv202338056:84 , srv202985091:84 , srv2056679767:84 , srv2068502106:84 , srv2073486758:84 , srv2111132818:84 , srv217829243:84 , srv225245512:84 , srv248910583:84 , srv324505696:84 , srv37073469:84 , srv410074558:84 , srv465690368:84 , srv502005008:84 , srv519811277:84 , srv608557487:84 , srv62437199:84 , srv670205242:84 , srv712624724:84 , srv722027430:84 , srv784201645:84 , srv788826737:84 , srv796489474:84 , srv802546454:84 , srv804476407:84 , srv817369744:84 , srv914026760:84 , srv917051432:84 , srv932625031:84 , srv951937966:84 , srv964140511:84 , srv982949874:84 , srv999637667:84 , srv1394793626:85 , srv1514431040:85 , srv1751601412:85 , srv1963537254:85 , srv484787487:85 , srv745726284:85 , srv1137072971:684 }
2023-10-08 11:57:03,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:03,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv712624724=83, srv1514431040=22, srv1863595173=50, srv1698753885=37, srv1932690537=55, srv465690368=74, srv1593026093=27, srv257264149=69, srv914026760=93, srv1683987373=34, srv1246285007=9, srv2111132818=65, srv1405837199=16, srv592038463=79, srv1753759082=41, srv2013124454=59, srv1817485837=46, srv1779004378=43, srv1725953848=38, srv1418415944=17, srv1421022123=18, srv917051432=94, srv62437199=81, srv804476407=91, srv1137072971=4, srv982949874=98, srv1376034173=12, srv1612716927=32, srv1979271972=57, srv2073486758=64, srv202985091=61, srv1330496928=11, srv1397938204=15, srv1234847252=8, srv1687833178=35, srv484787487=76, srv951937966=96, srv1565683344=24, srv1832909915=47, srv202338056=60, srv999637667=99, srv1746859422=39, srv248910583=68, srv1101720880=1, srv1875719440=53, srv964140511=97, srv324505696=70, srv1607171848=30, srv1751601412=40, srv185790356=49, srv1171260116=7, srv1582851939=25, srv784201645=86, srv1801542256=45, srv37073469=72, srv1455591196=20, srv802546454=90, srv1135625904=3, srv1800094483=44, srv1107503175=2, srv1520843572=23, srv608557487=80, srv175807952=42, srv1323311448=10, srv115428851=6, srv1502611692=21, srv1910519708=54, srv1610221893=31, srv722027430=84, srv200700916=58, srv932625031=95, srv745726284=85, srv1841176509=48, srv471724878=75, srv1429762742=19, srv217829243=66, srv788826737=87, srv187057400=52, srv339098536=71, srv2068502106=63, srv1151807906=5, srv410074558=73, srv790003351=88, srv1963537254=56, srv1697026619=36, srv670205242=82, srv519811277=78, srv1621277365=33, srv1588436854=26, srv1864447003=51, srv225245512=67, srv817369744=92, srv1599295152=29, srv796489474=89, srv1394793626=14, srv2056679767=62, srv1595102154=28, srv1390536382=13, srv502005008=77, srv1005233533=0} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:03,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:03,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:03,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:03,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:03,603 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:03,604 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:03,605 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-08 11:57:03,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-08 11:57:03,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-08 11:57:03,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-08 11:57:03,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-08 11:57:03,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-08 11:57:03,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-08 11:57:03,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-08 11:57:03,689 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,689 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv2013124454:83 , srv257264149:83 , srv339098536:83 , srv471724878:83 , srv592038463:83 , srv790003351:83 , srv1005233533:84 , srv1101720880:84 , srv1107503175:84 , srv1135625904:84 , srv1151807906:84 , srv115428851:84 , srv1171260116:84 , srv1234847252:84 , srv1246285007:84 , srv1323311448:84 , srv1330496928:84 , srv1376034173:84 , srv1390536382:84 , srv1397938204:84 , srv1405837199:84 , srv1418415944:84 , srv1421022123:84 , srv1429762742:84 , srv1455591196:84 , srv1502611692:84 , srv1520843572:84 , srv1565683344:84 , srv1582851939:84 , srv1588436854:84 , srv1593026093:84 , srv1595102154:84 , srv1599295152:84 , srv1607171848:84 , srv1610221893:84 , srv1612716927:84 , srv1621277365:84 , srv1683987373:84 , srv1687833178:84 , srv1697026619:84 , srv1698753885:84 , srv1725953848:84 , srv1746859422:84 , srv1753759082:84 , srv175807952:84 , srv1779004378:84 , srv1800094483:84 , srv1801542256:84 , srv1817485837:84 , srv1832909915:84 , srv1841176509:84 , srv185790356:84 , srv1863595173:84 , srv1864447003:84 , srv187057400:84 , srv1875719440:84 , srv1910519708:84 , srv1932690537:84 , srv1979271972:84 , srv200700916:84 , srv202338056:84 , srv202985091:84 , srv2056679767:84 , srv2068502106:84 , srv2073486758:84 , srv2111132818:84 , srv217829243:84 , srv225245512:84 , srv248910583:84 , srv324505696:84 , srv37073469:84 , srv410074558:84 , srv465690368:84 , srv502005008:84 , srv519811277:84 , srv608557487:84 , srv62437199:84 , srv670205242:84 , srv712624724:84 , srv722027430:84 , srv784201645:84 , srv788826737:84 , srv796489474:84 , srv802546454:84 , srv804476407:84 , srv817369744:84 , srv914026760:84 , srv917051432:84 , srv932625031:84 , srv951937966:84 , srv964140511:84 , srv982949874:84 , srv999637667:84 , srv1394793626:85 , srv1514431040:85 , srv1751601412:85 , srv1963537254:85 , srv484787487:85 , srv745726284:85 , srv1137072971:684 }
2023-10-08 11:57:03,713 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 227), ProcessCount=168 (was 168), AvailableMemoryMB=3430 (was 3917)
2023-10-08 11:57:03,719 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=168, AvailableMemoryMB=3430
2023-10-08 11:57:03,719 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-08 11:57:03,720 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-08 11:57:03,731 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004458266:84 , srv1092728690:84 , srv1170255247:84 , srv1194132964:84 , srv1240858456:84 , srv1341214242:84 , srv136249105:84 , srv1380494030:84 , srv1389763986:84 , srv1588938004:84 , srv1615385756:84 , srv1649689130:84 , srv1664726265:84 , srv1759743636:84 , srv1865880209:84 , srv1964000535:84 , srv2035248559:84 , srv218474610:84 , srv295201148:84 , srv326479309:84 , srv345797757:84 , srv556695923:84 , srv623752916:84 , srv6737294:84 , srv694481331:84 , srv767934973:84 , srv834945744:84 , srv881787318:84 , srv990149798:84 , srv1251101414:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-08 11:57:03,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:57:03,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv694481331=25, srv990149798=29, srv218474610=18, srv767934973=26, srv1092728690=1, srv326479309=20, srv623752916=23, srv1251101414=5, srv1964000535=16, srv136249105=7, srv1588938004=10, srv1004458266=0, srv1380494030=8, srv556695923=22, srv1389763986=9, srv345797757=21, srv2035248559=17, srv6737294=24, srv1240858456=4, srv1865880209=15, srv1664726265=13, srv1194132964=3, srv1649689130=12, srv881787318=28, srv295201148=19, srv834945744=27, srv1341214242=6, srv1615385756=11, srv1759743636=14, srv1170255247=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:57:03,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:57:03,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:57:03,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:57:03,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:57:03,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:57:03,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:57:03,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:57:03,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-08 11:57:03,769 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1126842338296594, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:57:03,769 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912548107605599, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1126842338296594, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-08 11:59:03,751 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9842595 different iterations.  Found a solution that moves 1841 regions; Going from a computed imbalance of 0.9912548107605599 to a new imbalance of 4.296985841319951E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6818518518518518, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:59:03,757 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1004458266:90 , srv1092728690:90 , srv1170255247:90 , srv1194132964:90 , srv1240858456:90 , srv1251101414:90 , srv1341214242:90 , srv136249105:90 , srv1380494030:90 , srv1389763986:90 , srv1588938004:90 , srv1615385756:90 , srv1649689130:90 , srv1664726265:90 , srv1759743636:90 , srv1865880209:90 , srv1964000535:90 , srv2035248559:90 , srv218474610:90 , srv295201148:90 , srv326479309:90 , srv345797757:90 , srv556695923:90 , srv623752916:90 , srv6737294:90 , srv694481331:90 , srv767934973:90 , srv834945744:90 , srv881787318:90 , srv990149798:90 }
2023-10-08 11:59:03,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-08 11:59:03,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv694481331=25, srv990149798=29, srv218474610=18, srv767934973=26, srv1092728690=1, srv326479309=20, srv623752916=23, srv1251101414=5, srv1964000535=16, srv136249105=7, srv1588938004=10, srv1004458266=0, srv1380494030=8, srv556695923=22, srv1389763986=9, srv345797757=21, srv2035248559=17, srv6737294=24, srv1240858456=4, srv1865880209=15, srv1664726265=13, srv1194132964=3, srv1649689130=12, srv881787318=28, srv295201148=19, srv834945744=27, srv1341214242=6, srv1615385756=11, srv1759743636=14, srv1170255247=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-08 11:59:03,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-08 11:59:03,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-08 11:59:03,772 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-08 11:59:03,772 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1004458266:90 , srv1092728690:90 , srv1170255247:90 , srv1194132964:90 , srv1240858456:90 , srv1251101414:90 , srv1341214242:90 , srv136249105:90 , srv1380494030:90 , srv1389763986:90 , srv1588938004:90 , srv1615385756:90 , srv1649689130:90 , srv1664726265:90 , srv1759743636:90 , srv1865880209:90 , srv1964000535:90 , srv2035248559:90 , srv218474610:90 , srv295201148:90 , srv326479309:90 , srv345797757:90 , srv556695923:90 , srv623752916:90 , srv6737294:90 , srv694481331:90 , srv767934973:90 , srv834945744:90 , srv881787318:90 , srv990149798:90 }
2023-10-08 11:59:03,782 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=139 (was 225), ProcessCount=163 (was 168), AvailableMemoryMB=4047 (was 3430) - AvailableMemoryMB LEAK? -