2023-11-28 03:53:14,266 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-28 03:53:14,620 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-28 03:53:14,742 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-28 03:53:14,812 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-28 03:53:14,819 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-28 03:53:14,833 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=172, AvailableMemoryMB=8330
2023-11-28 03:53:14,835 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-28 03:53:14,835 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-28 03:53:14,976 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv10091465:84 , srv1013334732:84 , srv1022888887:84 , srv102784035:84 , srv1056758820:84 , srv1118577935:84 , srv1142375744:84 , srv1147789887:84 , srv117118286:84 , srv1172768675:84 , srv1187489145:84 , srv1212296987:84 , srv128176201:84 , srv1284455143:84 , srv1341025300:84 , srv1352470472:84 , srv1370071592:84 , srv1399903025:84 , srv1412238309:84 , srv1437935600:84 , srv1445548918:84 , srv1448780506:84 , srv1453269997:84 , srv1459630437:84 , srv1468108685:84 , srv1529307061:84 , srv1578408171:84 , srv1587354696:84 , srv1622587607:84 , srv1626344791:84 , srv1646136954:84 , srv166086179:84 , srv1689839904:84 , srv171992675:84 , srv1757086479:84 , srv1759261049:84 , srv1779596379:84 , srv1823298817:84 , srv1854198976:84 , srv1862319964:84 , srv1875617467:84 , srv1885239993:84 , srv1891122664:84 , srv191319960:84 , srv1925721311:84 , srv1981818321:84 , srv1981972551:84 , srv1998420233:84 , srv2002851752:84 , srv2011204568:84 , srv2017224496:84 , srv2069887362:84 , srv2071156420:84 , srv2074268683:84 , srv2076840058:84 , srv2112616705:84 , srv213705988:84 , srv217081124:84 , srv26521698:84 , srv265952415:84 , srv297823323:84 , srv328604561:84 , srv349257845:84 , srv379084943:84 , srv395041143:84 , srv424179646:84 , srv433922039:84 , srv435267773:84 , srv484411656:84 , srv55737459:84 , srv564532842:84 , srv573100406:84 , srv587280575:84 , srv608547592:84 , srv612132422:84 , srv630136633:84 , srv633611618:84 , srv672502072:84 , srv681343652:84 , srv691826823:84 , srv702020870:84 , srv706989450:84 , srv717481074:84 , srv73023001:84 , srv732550477:84 , srv834737876:84 , srv873800476:84 , srv896277589:84 , srv89827774:84 , srv89876523:84 , srv901293434:84 , srv904577074:84 , srv916490440:84 , srv91821926:84 , srv931487731:84 , srv949272910:84 , srv953442670:84 , srv96201448:84 , srv974985877:84 , srv483402254:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-28 03:53:14,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:14,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1056758820=4, srv166086179=31, srv433922039=66, srv1445548918=20, srv89827774=89, srv1885239993=41, srv73023001=84, srv1399903025=17, srv102784035=3, srv1891122664=42, srv1529307061=25, srv395041143=64, srv1187489145=10, srv1823298817=37, srv672502072=78, srv2011204568=49, srv1854198976=38, srv587280575=73, srv612132422=75, srv717481074=83, srv1875617467=40, srv953442670=97, srv1118577935=5, srv691826823=80, srv1468108685=24, srv1626344791=29, srv1142375744=6, srv171992675=33, srv896277589=88, srv901293434=91, srv1759261049=35, srv681343652=79, srv2112616705=55, srv1013334732=1, srv633611618=77, srv706989450=82, srv2074268683=53, srv608547592=74, srv931487731=95, srv916490440=93, srv1578408171=26, srv1998420233=47, srv1172768675=9, srv1022888887=2, srv1412238309=18, srv484411656=69, srv128176201=12, srv904577074=92, srv564532842=71, srv1925721311=44, srv1448780506=21, srv949272910=96, srv630136633=76, srv424179646=65, srv297823323=60, srv1147789887=7, srv1646136954=30, srv379084943=63, srv1370071592=16, srv26521698=58, srv1341025300=14, srv1622587607=28, srv1352470472=15, srv2069887362=51, srv702020870=81, srv217081124=57, srv435267773=67, srv2076840058=54, srv2017224496=50, srv55737459=70, srv1981972551=46, srv1779596379=36, srv732550477=85, srv349257845=62, srv1453269997=22, srv328604561=61, srv834737876=86, srv96201448=98, srv1587354696=27, srv89876523=90, srv117118286=8, srv191319960=43, srv2071156420=52, srv483402254=68, srv265952415=59, srv213705988=56, srv1981818321=45, srv873800476=87, srv1459630437=23, srv1689839904=32, srv10091465=0, srv1862319964=39, srv91821926=94, srv1284455143=13, srv573100406=72, srv2002851752=48, srv1212296987=11, srv1437935600=19, srv974985877=99, srv1757086479=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:15,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:15,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:15,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:15,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:15,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:15,091 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 03:53:15,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 03:53:15,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 03:53:15,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 03:53:15,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 03:53:15,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 03:53:15,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 03:53:15,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 03:53:15,344 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:15,344 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 03:53:15,345 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 03:53:15,787 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 655 ms to try 5000 different iterations.  Found a solution that moves 4739 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1759605340771414. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04073370003441349, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056046317225336116, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5265555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04073370003441349, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:15,813 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv117118286:82 , srv1172768675:82 , srv1925721311:82 , srv55737459:82 , srv73023001:82 , srv1284455143:83 , srv1757086479:83 , srv1875617467:83 , srv1885239993:83 , srv1981818321:83 , srv2074268683:83 , srv424179646:83 , srv633611618:83 , srv1056758820:84 , srv1453269997:84 , srv1529307061:84 , srv1862319964:84 , srv2011204568:84 , srv2069887362:84 , srv217081124:84 , srv435267773:84 , srv484411656:84 , srv608547592:84 , srv681343652:84 , srv949272910:84 , srv96201448:84 , srv974985877:84 , srv1013334732:85 , srv1437935600:85 , srv1448780506:85 , srv1587354696:85 , srv1626344791:85 , srv1759261049:85 , srv1823298817:85 , srv2002851752:85 , srv2017224496:85 , srv26521698:85 , srv612132422:85 , srv672502072:85 , srv102784035:86 , srv1212296987:86 , srv1459630437:86 , srv1578408171:86 , srv1622587607:86 , srv1646136954:86 , srv1854198976:86 , srv1981972551:86 , srv265952415:86 , srv328604561:86 , srv349257845:86 , srv834737876:86 , srv89876523:86 , srv904577074:86 , srv916490440:86 , srv931487731:86 , srv1118577935:87 , srv1187489145:87 , srv1412238309:87 , srv171992675:87 , srv1891122664:87 , srv2071156420:87 , srv297823323:87 , srv564532842:87 , srv587280575:87 , srv702020870:87 , srv873800476:87 , srv89827774:87 , srv901293434:87 , srv953442670:87 , srv1022888887:88 , srv1142375744:88 , srv1147789887:88 , srv1341025300:88 , srv1399903025:88 , srv2076840058:88 , srv2112616705:88 , srv379084943:88 , srv395041143:88 , srv433922039:88 , srv717481074:88 , srv732550477:88 , srv1468108685:89 , srv166086179:89 , srv191319960:89 , srv1998420233:89 , srv213705988:89 , srv630136633:89 , srv691826823:89 , srv91821926:89 , srv10091465:90 , srv128176201:90 , srv1352470472:90 , srv706989450:90 , srv1445548918:91 , srv1779596379:91 , srv573100406:92 , srv1689839904:93 , srv896277589:94 , srv1370071592:95 , srv483402254:452 }
2023-11-28 03:53:15,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:15,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1056758820=4, srv166086179=31, srv433922039=66, srv1445548918=20, srv89827774=89, srv1885239993=41, srv73023001=84, srv1399903025=17, srv102784035=3, srv1891122664=42, srv1529307061=25, srv395041143=64, srv1187489145=10, srv1823298817=37, srv672502072=78, srv2011204568=49, srv1854198976=38, srv587280575=73, srv612132422=75, srv717481074=83, srv1875617467=40, srv953442670=97, srv1118577935=5, srv691826823=80, srv1468108685=24, srv1626344791=29, srv1142375744=6, srv171992675=33, srv896277589=88, srv901293434=91, srv1759261049=35, srv681343652=79, srv2112616705=55, srv1013334732=1, srv633611618=77, srv706989450=82, srv2074268683=53, srv608547592=74, srv931487731=95, srv916490440=93, srv1578408171=26, srv1998420233=47, srv1172768675=9, srv1022888887=2, srv1412238309=18, srv484411656=69, srv128176201=12, srv904577074=92, srv564532842=71, srv1925721311=44, srv1448780506=21, srv949272910=96, srv630136633=76, srv424179646=65, srv297823323=60, srv1147789887=7, srv1646136954=30, srv379084943=63, srv1370071592=16, srv26521698=58, srv1341025300=14, srv1622587607=28, srv1352470472=15, srv2069887362=51, srv702020870=81, srv217081124=57, srv435267773=67, srv2076840058=54, srv2017224496=50, srv55737459=70, srv1981972551=46, srv1779596379=36, srv732550477=85, srv349257845=62, srv1453269997=22, srv328604561=61, srv834737876=86, srv96201448=98, srv1587354696=27, srv89876523=90, srv117118286=8, srv191319960=43, srv2071156420=52, srv483402254=68, srv265952415=59, srv213705988=56, srv1981818321=45, srv873800476=87, srv1459630437=23, srv1689839904=32, srv10091465=0, srv1862319964=39, srv91821926=94, srv1284455143=13, srv573100406=72, srv2002851752=48, srv1212296987=11, srv1437935600=19, srv974985877=99, srv1757086479=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:15,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:15,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:15,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:15,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:15,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 03:53:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 03:53:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 03:53:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 03:53:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 03:53:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 03:53:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 03:53:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 03:53:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 03:53:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 03:53:16,026 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04073370003441349, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056046317225336116, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04073370003441349, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:16,027 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 03:53:16,027 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1759273508899029, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04073370003441349, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056046317225336116, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04073370003441349, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17091666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 03:53:16,294 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 412 ms to try 5000 different iterations.  Found a solution that moves 2550 regions; Going from a computed imbalance of 0.1759273508899029 to a new imbalance of 0.015754701163739112. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02149772983545737); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03133397807202561, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02149772983545737); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04541666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:16,311 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1370071592:77 , srv896277589:78 , srv1689839904:79 , srv10091465:80 , srv128176201:80 , srv573100406:80 , srv1352470472:82 , srv1445548918:82 , srv166086179:82 , srv1779596379:82 , srv2076840058:82 , srv213705988:82 , srv706989450:82 , srv873800476:82 , srv1399903025:83 , srv297823323:83 , srv691826823:83 , srv89876523:83 , srv1187489145:84 , srv1468108685:84 , srv171992675:84 , srv191319960:84 , srv1998420233:84 , srv395041143:84 , srv433922039:84 , srv587280575:84 , srv702020870:84 , srv717481074:84 , srv732550477:84 , srv89827774:84 , srv916490440:84 , srv91821926:84 , srv1022888887:85 , srv102784035:85 , srv1142375744:85 , srv1147789887:85 , srv1587354696:85 , srv2071156420:85 , srv2112616705:85 , srv26521698:85 , srv630136633:85 , srv672502072:85 , srv931487731:85 , srv953442670:85 , srv1013334732:86 , srv1118577935:86 , srv1341025300:86 , srv1412238309:86 , srv1459630437:86 , srv1622587607:86 , srv1646136954:86 , srv1759261049:86 , srv1854198976:86 , srv217081124:86 , srv265952415:86 , srv349257845:86 , srv564532842:86 , srv612132422:86 , srv834737876:86 , srv1212296987:87 , srv1284455143:87 , srv1448780506:87 , srv1453269997:87 , srv1626344791:87 , srv1885239993:87 , srv1891122664:87 , srv1981972551:87 , srv2017224496:87 , srv2069887362:87 , srv328604561:87 , srv901293434:87 , srv904577074:87 , srv949272910:87 , srv96201448:87 , srv1529307061:88 , srv1578408171:88 , srv1823298817:88 , srv2002851752:88 , srv435267773:88 , srv484411656:88 , srv633611618:88 , srv681343652:88 , srv73023001:88 , srv974985877:88 , srv1437935600:89 , srv1875617467:89 , srv1925721311:89 , srv1981818321:89 , srv2011204568:89 , srv1056758820:90 , srv1757086479:90 , srv1862319964:90 , srv2074268683:90 , srv379084943:90 , srv117118286:91 , srv1172768675:91 , srv424179646:92 , srv608547592:92 , srv55737459:93 , srv483402254:513 }
2023-11-28 03:53:16,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:16,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1056758820=4, srv166086179=31, srv433922039=66, srv1445548918=20, srv89827774=89, srv1885239993=41, srv73023001=84, srv1399903025=17, srv102784035=3, srv1891122664=42, srv1529307061=25, srv395041143=64, srv1187489145=10, srv1823298817=37, srv672502072=78, srv2011204568=49, srv1854198976=38, srv587280575=73, srv612132422=75, srv717481074=83, srv1875617467=40, srv953442670=97, srv1118577935=5, srv691826823=80, srv1468108685=24, srv1626344791=29, srv1142375744=6, srv171992675=33, srv896277589=88, srv901293434=91, srv1759261049=35, srv681343652=79, srv2112616705=55, srv1013334732=1, srv633611618=77, srv706989450=82, srv2074268683=53, srv608547592=74, srv931487731=95, srv916490440=93, srv1578408171=26, srv1998420233=47, srv1172768675=9, srv1022888887=2, srv1412238309=18, srv484411656=69, srv128176201=12, srv904577074=92, srv564532842=71, srv1925721311=44, srv1448780506=21, srv949272910=96, srv630136633=76, srv424179646=65, srv297823323=60, srv1147789887=7, srv1646136954=30, srv379084943=63, srv1370071592=16, srv26521698=58, srv1341025300=14, srv1622587607=28, srv1352470472=15, srv2069887362=51, srv702020870=81, srv217081124=57, srv435267773=67, srv2076840058=54, srv2017224496=50, srv55737459=70, srv1981972551=46, srv1779596379=36, srv732550477=85, srv349257845=62, srv1453269997=22, srv328604561=61, srv834737876=86, srv96201448=98, srv1587354696=27, srv89876523=90, srv117118286=8, srv191319960=43, srv2071156420=52, srv483402254=68, srv265952415=59, srv213705988=56, srv1981818321=45, srv873800476=87, srv1459630437=23, srv1689839904=32, srv10091465=0, srv1862319964=39, srv91821926=94, srv1284455143=13, srv573100406=72, srv2002851752=48, srv1212296987=11, srv1437935600=19, srv974985877=99, srv1757086479=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:16,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:16,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:16,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:16,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:16,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 03:53:16,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 03:53:16,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 03:53:16,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 03:53:16,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 03:53:16,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 03:53:16,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 03:53:16,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 03:53:16,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 03:53:16,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 03:53:16,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 03:53:16,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 03:53:16,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 03:53:16,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 03:53:16,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 03:53:16,346 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 03:53:16,506 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02149772983545737); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03133397807202561, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02149772983545737); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04541666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:16,506 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 03:53:16,506 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015736845682106253, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02149772983545737); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03133397807202561, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02149772983545737); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04541666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 03:53:16,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 523 ms to try 5000 different iterations.  Found a solution that moves 971 regions; Going from a computed imbalance of 0.015736845682106253 to a new imbalance of 6.534189880794129E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006557276876487454); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00663291117550788); MoveCostFunction : (multiplier=7.0, imbalance=0.1078888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006557276876487454); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:16,895 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv379084943:80 , srv608547592:82 , srv1056758820:83 , srv1212296987:83 , srv1445548918:83 , srv1891122664:83 , srv1998420233:83 , srv424179646:83 , srv55737459:83 , srv630136633:83 , srv901293434:83 , srv1022888887:84 , srv1118577935:84 , srv1172768675:84 , srv1341025300:84 , srv1370071592:84 , srv1437935600:84 , srv1468108685:84 , srv1578408171:84 , srv1779596379:84 , srv1862319964:84 , srv191319960:84 , srv2112616705:84 , srv904577074:84 , srv91821926:84 , srv1142375744:85 , srv117118286:85 , srv1352470472:85 , srv1399903025:85 , srv1412238309:85 , srv1448780506:85 , srv1459630437:85 , srv1622587607:85 , srv1626344791:85 , srv1646136954:85 , srv1689839904:85 , srv1757086479:85 , srv1823298817:85 , srv1854198976:85 , srv1875617467:85 , srv1981972551:85 , srv2002851752:85 , srv2011204568:85 , srv2069887362:85 , srv328604561:85 , srv349257845:85 , srv433922039:85 , srv435267773:85 , srv484411656:85 , srv564532842:85 , srv612132422:85 , srv681343652:85 , srv691826823:85 , srv953442670:85 , srv974985877:85 , srv1013334732:86 , srv1147789887:86 , srv1529307061:86 , srv171992675:86 , srv1759261049:86 , srv1925721311:86 , srv1981818321:86 , srv2017224496:86 , srv2071156420:86 , srv2074268683:86 , srv213705988:86 , srv265952415:86 , srv573100406:86 , srv587280575:86 , srv633611618:86 , srv706989450:86 , srv717481074:86 , srv834737876:86 , srv896277589:86 , srv931487731:86 , srv949272910:86 , srv96201448:86 , srv10091465:87 , srv102784035:87 , srv1187489145:87 , srv128176201:87 , srv1453269997:87 , srv1587354696:87 , srv166086179:87 , srv2076840058:87 , srv26521698:87 , srv297823323:87 , srv702020870:87 , srv73023001:87 , srv732550477:87 , srv89827774:87 , srv916490440:87 , srv1284455143:88 , srv1885239993:88 , srv217081124:88 , srv395041143:88 , srv672502072:88 , srv89876523:88 , srv873800476:89 , srv483402254:551 }
2023-11-28 03:53:16,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:16,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1056758820=4, srv166086179=31, srv433922039=66, srv1445548918=20, srv89827774=89, srv1885239993=41, srv73023001=84, srv1399903025=17, srv102784035=3, srv1891122664=42, srv1529307061=25, srv395041143=64, srv1187489145=10, srv1823298817=37, srv672502072=78, srv2011204568=49, srv1854198976=38, srv587280575=73, srv612132422=75, srv717481074=83, srv1875617467=40, srv953442670=97, srv1118577935=5, srv691826823=80, srv1468108685=24, srv1626344791=29, srv1142375744=6, srv171992675=33, srv896277589=88, srv901293434=91, srv1759261049=35, srv681343652=79, srv2112616705=55, srv1013334732=1, srv633611618=77, srv706989450=82, srv2074268683=53, srv608547592=74, srv931487731=95, srv916490440=93, srv1578408171=26, srv1998420233=47, srv1172768675=9, srv1022888887=2, srv1412238309=18, srv484411656=69, srv128176201=12, srv904577074=92, srv564532842=71, srv1925721311=44, srv1448780506=21, srv949272910=96, srv630136633=76, srv424179646=65, srv297823323=60, srv1147789887=7, srv1646136954=30, srv379084943=63, srv1370071592=16, srv26521698=58, srv1341025300=14, srv1622587607=28, srv1352470472=15, srv2069887362=51, srv702020870=81, srv217081124=57, srv435267773=67, srv2076840058=54, srv2017224496=50, srv55737459=70, srv1981972551=46, srv1779596379=36, srv732550477=85, srv349257845=62, srv1453269997=22, srv328604561=61, srv834737876=86, srv96201448=98, srv1587354696=27, srv89876523=90, srv117118286=8, srv191319960=43, srv2071156420=52, srv483402254=68, srv265952415=59, srv213705988=56, srv1981818321=45, srv873800476=87, srv1459630437=23, srv1689839904=32, srv10091465=0, srv1862319964=39, srv91821926=94, srv1284455143=13, srv573100406=72, srv2002851752=48, srv1212296987=11, srv1437935600=19, srv974985877=99, srv1757086479=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:16,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:16,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:16,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 03:53:16,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 03:53:16,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 03:53:16,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 03:53:16,916 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 03:53:16,917 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 03:53:16,918 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 03:53:16,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 03:53:16,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 03:53:16,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 03:53:16,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 03:53:16,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 03:53:16,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 03:53:17,046 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006557276876487454); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00663291117550788); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006557276876487454); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:17,046 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 03:53:17,047 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.466199007596058E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006557276876487454); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00663291117550788); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006557276876487454); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 03:53:17,428 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 493 ms to try 5000 different iterations.  Found a solution that moves 250 regions; Going from a computed imbalance of 6.466199007596058E-4 to a new imbalance of 3.493953601643618E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.027777777777777776, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:17,429 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv395041143:82 , srv1147789887:83 , srv2002851752:83 , srv732550477:83 , srv1056758820:84 , srv1142375744:84 , srv117118286:84 , srv1187489145:84 , srv1284455143:84 , srv1341025300:84 , srv1412238309:84 , srv1437935600:84 , srv1453269997:84 , srv1468108685:84 , srv1578408171:84 , srv166086179:84 , srv1757086479:84 , srv1759261049:84 , srv1823298817:84 , srv1862319964:84 , srv1875617467:84 , srv1885239993:84 , srv1925721311:84 , srv1981818321:84 , srv1981972551:84 , srv2011204568:84 , srv2017224496:84 , srv2071156420:84 , srv2074268683:84 , srv265952415:84 , srv379084943:84 , srv424179646:84 , srv55737459:84 , srv564532842:84 , srv573100406:84 , srv608547592:84 , srv630136633:84 , srv672502072:84 , srv702020870:84 , srv706989450:84 , srv717481074:84 , srv834737876:84 , srv873800476:84 , srv896277589:84 , srv89827774:84 , srv953442670:84 , srv10091465:85 , srv1013334732:85 , srv1022888887:85 , srv102784035:85 , srv1118577935:85 , srv1172768675:85 , srv128176201:85 , srv1352470472:85 , srv1448780506:85 , srv1459630437:85 , srv1529307061:85 , srv1587354696:85 , srv1622587607:85 , srv1626344791:85 , srv1646136954:85 , srv1689839904:85 , srv171992675:85 , srv1779596379:85 , srv1854198976:85 , srv1891122664:85 , srv191319960:85 , srv2076840058:85 , srv2112616705:85 , srv213705988:85 , srv217081124:85 , srv26521698:85 , srv297823323:85 , srv328604561:85 , srv349257845:85 , srv433922039:85 , srv435267773:85 , srv484411656:85 , srv587280575:85 , srv633611618:85 , srv681343652:85 , srv691826823:85 , srv73023001:85 , srv89876523:85 , srv901293434:85 , srv904577074:85 , srv916490440:85 , srv91821926:85 , srv931487731:85 , srv949272910:85 , srv974985877:85 , srv1370071592:86 , srv1399903025:86 , srv1445548918:86 , srv1998420233:86 , srv2069887362:86 , srv96201448:86 , srv1212296987:87 , srv612132422:87 , srv483402254:626 }
2023-11-28 03:53:17,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:17,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1056758820=4, srv166086179=31, srv433922039=66, srv1445548918=20, srv89827774=89, srv1885239993=41, srv73023001=84, srv1399903025=17, srv102784035=3, srv1891122664=42, srv1529307061=25, srv395041143=64, srv1187489145=10, srv1823298817=37, srv672502072=78, srv2011204568=49, srv1854198976=38, srv587280575=73, srv612132422=75, srv717481074=83, srv1875617467=40, srv953442670=97, srv1118577935=5, srv691826823=80, srv1468108685=24, srv1626344791=29, srv1142375744=6, srv171992675=33, srv896277589=88, srv901293434=91, srv1759261049=35, srv681343652=79, srv2112616705=55, srv1013334732=1, srv633611618=77, srv706989450=82, srv2074268683=53, srv608547592=74, srv931487731=95, srv916490440=93, srv1578408171=26, srv1998420233=47, srv1172768675=9, srv1022888887=2, srv1412238309=18, srv484411656=69, srv128176201=12, srv904577074=92, srv564532842=71, srv1925721311=44, srv1448780506=21, srv949272910=96, srv630136633=76, srv424179646=65, srv297823323=60, srv1147789887=7, srv1646136954=30, srv379084943=63, srv1370071592=16, srv26521698=58, srv1341025300=14, srv1622587607=28, srv1352470472=15, srv2069887362=51, srv702020870=81, srv217081124=57, srv435267773=67, srv2076840058=54, srv2017224496=50, srv55737459=70, srv1981972551=46, srv1779596379=36, srv732550477=85, srv349257845=62, srv1453269997=22, srv328604561=61, srv834737876=86, srv96201448=98, srv1587354696=27, srv89876523=90, srv117118286=8, srv191319960=43, srv2071156420=52, srv483402254=68, srv265952415=59, srv213705988=56, srv1981818321=45, srv873800476=87, srv1459630437=23, srv1689839904=32, srv10091465=0, srv1862319964=39, srv91821926=94, srv1284455143=13, srv573100406=72, srv2002851752=48, srv1212296987=11, srv1437935600=19, srv974985877=99, srv1757086479=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:17,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:17,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:17,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:17,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:17,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:17,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:17,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 03:53:17,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 03:53:17,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 03:53:17,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 03:53:17,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 03:53:17,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 03:53:17,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 03:53:17,618 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:17,618 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-28 03:53:17,619 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.3188998601449776E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-28 03:53:17,961 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 479 ms to try 5000 different iterations.  Found a solution that moves 22 regions; Going from a computed imbalance of 3.3188998601449776E-5 to a new imbalance of 7.541804792831362E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0024444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:17,961 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv102784035:83 , srv1212296987:83 , srv1529307061:83 , srv2074268683:83 , srv217081124:83 , srv328604561:83 , srv612132422:83 , srv96201448:83 , srv10091465:84 , srv1013334732:84 , srv1022888887:84 , srv1118577935:84 , srv1142375744:84 , srv1147789887:84 , srv117118286:84 , srv1172768675:84 , srv1187489145:84 , srv128176201:84 , srv1284455143:84 , srv1341025300:84 , srv1352470472:84 , srv1370071592:84 , srv1399903025:84 , srv1412238309:84 , srv1437935600:84 , srv1445548918:84 , srv1448780506:84 , srv1453269997:84 , srv1459630437:84 , srv1578408171:84 , srv1587354696:84 , srv1622587607:84 , srv1626344791:84 , srv1646136954:84 , srv166086179:84 , srv1689839904:84 , srv171992675:84 , srv1757086479:84 , srv1779596379:84 , srv1823298817:84 , srv1854198976:84 , srv1862319964:84 , srv1885239993:84 , srv1891122664:84 , srv191319960:84 , srv1981818321:84 , srv1981972551:84 , srv1998420233:84 , srv2011204568:84 , srv2017224496:84 , srv2069887362:84 , srv2071156420:84 , srv2076840058:84 , srv2112616705:84 , srv213705988:84 , srv26521698:84 , srv265952415:84 , srv297823323:84 , srv349257845:84 , srv379084943:84 , srv395041143:84 , srv424179646:84 , srv433922039:84 , srv435267773:84 , srv484411656:84 , srv55737459:84 , srv564532842:84 , srv573100406:84 , srv587280575:84 , srv608547592:84 , srv633611618:84 , srv672502072:84 , srv681343652:84 , srv691826823:84 , srv702020870:84 , srv706989450:84 , srv717481074:84 , srv73023001:84 , srv732550477:84 , srv834737876:84 , srv873800476:84 , srv896277589:84 , srv89827774:84 , srv89876523:84 , srv901293434:84 , srv904577074:84 , srv916490440:84 , srv91821926:84 , srv931487731:84 , srv949272910:84 , srv974985877:84 , srv1056758820:85 , srv1468108685:85 , srv1759261049:85 , srv1875617467:85 , srv1925721311:85 , srv2002851752:85 , srv630136633:85 , srv953442670:85 , srv483402254:684 }
2023-11-28 03:53:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1056758820=4, srv166086179=31, srv433922039=66, srv1445548918=20, srv89827774=89, srv1885239993=41, srv73023001=84, srv1399903025=17, srv102784035=3, srv1891122664=42, srv1529307061=25, srv395041143=64, srv1187489145=10, srv1823298817=37, srv672502072=78, srv2011204568=49, srv1854198976=38, srv587280575=73, srv612132422=75, srv717481074=83, srv1875617467=40, srv953442670=97, srv1118577935=5, srv691826823=80, srv1468108685=24, srv1626344791=29, srv1142375744=6, srv171992675=33, srv896277589=88, srv901293434=91, srv1759261049=35, srv681343652=79, srv2112616705=55, srv1013334732=1, srv633611618=77, srv706989450=82, srv2074268683=53, srv608547592=74, srv931487731=95, srv916490440=93, srv1578408171=26, srv1998420233=47, srv1172768675=9, srv1022888887=2, srv1412238309=18, srv484411656=69, srv128176201=12, srv904577074=92, srv564532842=71, srv1925721311=44, srv1448780506=21, srv949272910=96, srv630136633=76, srv424179646=65, srv297823323=60, srv1147789887=7, srv1646136954=30, srv379084943=63, srv1370071592=16, srv26521698=58, srv1341025300=14, srv1622587607=28, srv1352470472=15, srv2069887362=51, srv702020870=81, srv217081124=57, srv435267773=67, srv2076840058=54, srv2017224496=50, srv55737459=70, srv1981972551=46, srv1779596379=36, srv732550477=85, srv349257845=62, srv1453269997=22, srv328604561=61, srv834737876=86, srv96201448=98, srv1587354696=27, srv89876523=90, srv117118286=8, srv191319960=43, srv2071156420=52, srv483402254=68, srv265952415=59, srv213705988=56, srv1981818321=45, srv873800476=87, srv1459630437=23, srv1689839904=32, srv10091465=0, srv1862319964=39, srv91821926=94, srv1284455143=13, srv573100406=72, srv2002851752=48, srv1212296987=11, srv1437935600=19, srv974985877=99, srv1757086479=34} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:17,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:17,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:17,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-28 03:53:17,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-28 03:53:17,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-28 03:53:17,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-28 03:53:17,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-28 03:53:17,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-28 03:53:17,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-28 03:53:17,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-28 03:53:17,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-28 03:53:17,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-28 03:53:18,152 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:18,153 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv102784035:83 , srv1212296987:83 , srv1529307061:83 , srv2074268683:83 , srv217081124:83 , srv328604561:83 , srv612132422:83 , srv96201448:83 , srv10091465:84 , srv1013334732:84 , srv1022888887:84 , srv1118577935:84 , srv1142375744:84 , srv1147789887:84 , srv117118286:84 , srv1172768675:84 , srv1187489145:84 , srv128176201:84 , srv1284455143:84 , srv1341025300:84 , srv1352470472:84 , srv1370071592:84 , srv1399903025:84 , srv1412238309:84 , srv1437935600:84 , srv1445548918:84 , srv1448780506:84 , srv1453269997:84 , srv1459630437:84 , srv1578408171:84 , srv1587354696:84 , srv1622587607:84 , srv1626344791:84 , srv1646136954:84 , srv166086179:84 , srv1689839904:84 , srv171992675:84 , srv1757086479:84 , srv1779596379:84 , srv1823298817:84 , srv1854198976:84 , srv1862319964:84 , srv1885239993:84 , srv1891122664:84 , srv191319960:84 , srv1981818321:84 , srv1981972551:84 , srv1998420233:84 , srv2011204568:84 , srv2017224496:84 , srv2069887362:84 , srv2071156420:84 , srv2076840058:84 , srv2112616705:84 , srv213705988:84 , srv26521698:84 , srv265952415:84 , srv297823323:84 , srv349257845:84 , srv379084943:84 , srv395041143:84 , srv424179646:84 , srv433922039:84 , srv435267773:84 , srv484411656:84 , srv55737459:84 , srv564532842:84 , srv573100406:84 , srv587280575:84 , srv608547592:84 , srv633611618:84 , srv672502072:84 , srv681343652:84 , srv691826823:84 , srv702020870:84 , srv706989450:84 , srv717481074:84 , srv73023001:84 , srv732550477:84 , srv834737876:84 , srv873800476:84 , srv896277589:84 , srv89827774:84 , srv89876523:84 , srv901293434:84 , srv904577074:84 , srv916490440:84 , srv91821926:84 , srv931487731:84 , srv949272910:84 , srv974985877:84 , srv1056758820:85 , srv1468108685:85 , srv1759261049:85 , srv1875617467:85 , srv1925721311:85 , srv2002851752:85 , srv630136633:85 , srv953442670:85 , srv483402254:684 }
2023-11-28 03:53:18,191 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=323 (was 323), ProcessCount=172 (was 172), AvailableMemoryMB=7388 (was 8330)
2023-11-28 03:53:18,198 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=172, AvailableMemoryMB=7388
2023-11-28 03:53:18,199 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-28 03:53:18,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-28 03:53:18,212 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1008158492:84 , srv1025385791:84 , srv1026761235:84 , srv1055807880:84 , srv1082990107:84 , srv1091626161:84 , srv121569744:84 , srv122003473:84 , srv1247920811:84 , srv1414965971:84 , srv1514134202:84 , srv163562636:84 , srv1698037248:84 , srv1838057937:84 , srv184407574:84 , srv20360209:84 , srv251736285:84 , srv335130163:84 , srv520925077:84 , srv570488835:84 , srv629642592:84 , srv689533269:84 , srv693450388:84 , srv756554308:84 , srv768818716:84 , srv818410321:84 , srv94212168:84 , srv949913242:84 , srv971184024:84 , srv220521510:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-28 03:53:18,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:53:18,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1247920811=8, srv1838057937=13, srv251736285=17, srv1082990107=4, srv629642592=21, srv1026761235=2, srv693450388=23, srv1414965971=9, srv768818716=25, srv184407574=14, srv949913242=28, srv1091626161=5, srv971184024=29, srv20360209=15, srv1055807880=3, srv1514134202=10, srv520925077=19, srv1025385791=1, srv163562636=11, srv1008158492=0, srv689533269=22, srv1698037248=12, srv94212168=27, srv121569744=6, srv220521510=16, srv335130163=18, srv818410321=26, srv122003473=7, srv570488835=20, srv756554308=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:53:18,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:53:18,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:53:18,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:53:18,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:53:18,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:53:18,228 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:53:18,229 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:53:18,230 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:53:18,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-28 03:53:18,257 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0938990660647228, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:53:18,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912488916155364, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0938990660647228, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-28 03:55:18,236 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6024626 different iterations.  Found a solution that moves 1848 regions; Going from a computed imbalance of 0.9912488916155364 to a new imbalance of 4.313324190526491E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6844444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:55:18,243 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1008158492:90 , srv1025385791:90 , srv1026761235:90 , srv1055807880:90 , srv1082990107:90 , srv1091626161:90 , srv121569744:90 , srv122003473:90 , srv1247920811:90 , srv1414965971:90 , srv1514134202:90 , srv163562636:90 , srv1698037248:90 , srv1838057937:90 , srv184407574:90 , srv20360209:90 , srv220521510:90 , srv251736285:90 , srv335130163:90 , srv520925077:90 , srv570488835:90 , srv629642592:90 , srv689533269:90 , srv693450388:90 , srv756554308:90 , srv768818716:90 , srv818410321:90 , srv94212168:90 , srv949913242:90 , srv971184024:90 }
2023-11-28 03:55:18,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-28 03:55:18,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1247920811=8, srv1838057937=13, srv251736285=17, srv1082990107=4, srv629642592=21, srv1026761235=2, srv693450388=23, srv1414965971=9, srv768818716=25, srv184407574=14, srv949913242=28, srv1091626161=5, srv971184024=29, srv20360209=15, srv1055807880=3, srv1514134202=10, srv520925077=19, srv1025385791=1, srv163562636=11, srv1008158492=0, srv689533269=22, srv1698037248=12, srv94212168=27, srv121569744=6, srv220521510=16, srv335130163=18, srv818410321=26, srv122003473=7, srv570488835=20, srv756554308=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-28 03:55:18,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-28 03:55:18,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-28 03:55:18,249 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-28 03:55:18,266 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-28 03:55:18,266 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1008158492:90 , srv1025385791:90 , srv1026761235:90 , srv1055807880:90 , srv1082990107:90 , srv1091626161:90 , srv121569744:90 , srv122003473:90 , srv1247920811:90 , srv1414965971:90 , srv1514134202:90 , srv163562636:90 , srv1698037248:90 , srv1838057937:90 , srv184407574:90 , srv20360209:90 , srv220521510:90 , srv251736285:90 , srv335130163:90 , srv520925077:90 , srv570488835:90 , srv629642592:90 , srv689533269:90 , srv693450388:90 , srv756554308:90 , srv768818716:90 , srv818410321:90 , srv94212168:90 , srv949913242:90 , srv971184024:90 }
2023-11-28 03:55:18,281 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=196 (was 323), ProcessCount=168 (was 172), AvailableMemoryMB=6845 (was 7388)