2023-10-05 04:53:23,484 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-05 04:53:23,780 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-05 04:53:23,885 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-05 04:53:23,950 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-05 04:53:23,958 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-05 04:53:23,971 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=282, ProcessCount=169, AvailableMemoryMB=7241
2023-10-05 04:53:23,973 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-05 04:53:23,973 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-05 04:53:24,128 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001707305:84 , srv1014696923:84 , srv1037232168:84 , srv1064076002:84 , srv107607601:84 , srv1084268065:84 , srv1091777065:84 , srv110721899:84 , srv111382681:84 , srv1124742268:84 , srv1132747502:84 , srv115159320:84 , srv1181617906:84 , srv1206406752:84 , srv1222150891:84 , srv124289088:84 , srv1243644938:84 , srv1250558867:84 , srv1263107680:84 , srv1269081156:84 , srv1293170057:84 , srv1296015349:84 , srv1329407150:84 , srv138687389:84 , srv1401627029:84 , srv1406625017:84 , srv1410826442:84 , srv1414009685:84 , srv1432364168:84 , srv1432933960:84 , srv1445391656:84 , srv1450327863:84 , srv1467541395:84 , srv1511096967:84 , srv1534922016:84 , srv1569868387:84 , srv1611684327:84 , srv1625743213:84 , srv167011581:84 , srv1708277138:84 , srv1763733514:84 , srv180464267:84 , srv1825375263:84 , srv1827325017:84 , srv1828559085:84 , srv1854799754:84 , srv1874429614:84 , srv190310081:84 , srv1964482910:84 , srv1981885335:84 , srv1984811680:84 , srv2000396980:84 , srv2024203262:84 , srv2035090562:84 , srv2054649602:84 , srv2083727223:84 , srv2088149140:84 , srv2089907916:84 , srv2110836667:84 , srv217935300:84 , srv227956873:84 , srv264971717:84 , srv319378747:84 , srv32880409:84 , srv345968430:84 , srv351101530:84 , srv369734436:84 , srv403036198:84 , srv406172948:84 , srv416527857:84 , srv482411159:84 , srv506221545:84 , srv514634149:84 , srv532666590:84 , srv538552992:84 , srv555647052:84 , srv599908824:84 , srv63169025:84 , srv645199608:84 , srv649638533:84 , srv660140566:84 , srv699197785:84 , srv704720413:84 , srv706393907:84 , srv731343367:84 , srv74016846:84 , srv755010207:84 , srv78420693:84 , srv793888831:84 , srv79715193:84 , srv799991109:84 , srv87019332:84 , srv871045212:84 , srv911933899:84 , srv922772861:84 , srv961321317:84 , srv969394298:84 , srv995702068:84 , srv996058553:84 , srv1979415199:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-05 04:53:24,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:24,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1269081156=19, srv996058553=99, srv599908824=77, srv1445391656=30, srv995702068=98, srv345968430=65, srv506221545=72, srv1569868387=35, srv1432364168=28, srv1250558867=17, srv1329407150=22, srv1611684327=36, srv731343367=85, srv1132747502=10, srv1964482910=48, srv1511096967=33, srv1625743213=37, srv1828559085=44, srv1534922016=34, srv1293170057=20, srv403036198=68, srv63169025=78, srv871045212=93, srv406172948=69, srv1854799754=45, srv704720413=83, srv1014696923=1, srv227956873=61, srv110721899=7, srv538552992=75, srv2035090562=54, srv87019332=92, srv217935300=60, srv922772861=95, srv1432933960=29, srv1206406752=13, srv699197785=82, srv1037232168=2, srv2088149140=57, srv1124742268=9, srv961321317=96, srv1763733514=40, srv1827325017=43, srv78420693=88, srv1243644938=16, srv138687389=23, srv482411159=71, srv2054649602=55, srv180464267=41, srv1406625017=25, srv1222150891=14, srv2083727223=56, srv1467541395=32, srv1263107680=18, srv2024203262=53, srv74016846=86, srv107607601=4, srv514634149=73, srv1401627029=24, srv799991109=91, srv124289088=15, srv1979415199=49, srv1981885335=50, srv532666590=74, srv660140566=81, srv32880409=64, srv111382681=8, srv79715193=90, srv911933899=94, srv416527857=70, srv555647052=76, srv1984811680=51, srv264971717=62, srv2089907916=58, srv649638533=80, srv115159320=11, srv1708277138=39, srv1064076002=3, srv706393907=84, srv1414009685=27, srv319378747=63, srv1091777065=6, srv645199608=79, srv1874429614=46, srv2110836667=59, srv2000396980=52, srv755010207=87, srv969394298=97, srv1410826442=26, srv793888831=89, srv1825375263=42, srv1296015349=21, srv1450327863=31, srv351101530=66, srv1001707305=0, srv167011581=38, srv369734436=67, srv1181617906=12, srv1084268065=5, srv190310081=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:24,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:24,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:24,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:24,233 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:24,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:24,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:24,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:24,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:24,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:24,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:24,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:24,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:24,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:24,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:24,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:24,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:24,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:24,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:24,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:24,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:24,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:24,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:24,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:24,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:24,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:24,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:24,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:24,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-05 04:53:24,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-05 04:53:24,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-05 04:53:24,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-05 04:53:24,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-05 04:53:24,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-05 04:53:24,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-05 04:53:24,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-05 04:53:24,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-05 04:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-05 04:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-05 04:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-05 04:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-05 04:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-05 04:53:24,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-05 04:53:24,242 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-05 04:53:24,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-05 04:53:24,244 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-05 04:53:24,245 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-05 04:53:24,246 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-05 04:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-05 04:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-05 04:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-05 04:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-05 04:53:24,247 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-05 04:53:24,248 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-05 04:53:24,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:24,469 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-05 04:53:24,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-05 04:53:24,848 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 578 ms to try 5000 different iterations.  Found a solution that moves 4757 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17505053154171518. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03701918761806819, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05783214225575611, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5285555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03701918761806819, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16966666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:24,863 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1243644938:81 , srv1874429614:81 , srv190310081:82 , srv115159320:83 , srv1296015349:83 , srv1414009685:83 , srv2089907916:83 , srv78420693:83 , srv110721899:84 , srv124289088:84 , srv167011581:84 , srv2000396980:84 , srv345968430:84 , srv555647052:84 , srv1206406752:85 , srv1269081156:85 , srv138687389:85 , srv1445391656:85 , srv1611684327:85 , srv1763733514:85 , srv319378747:85 , srv32880409:85 , srv369734436:85 , srv532666590:85 , srv645199608:85 , srv704720413:85 , srv706393907:85 , srv1001707305:86 , srv1084268065:86 , srv1091777065:86 , srv111382681:86 , srv1124742268:86 , srv1181617906:86 , srv1432364168:86 , srv1450327863:86 , srv1511096967:86 , srv1708277138:86 , srv180464267:86 , srv1825375263:86 , srv2035090562:86 , srv2054649602:86 , srv217935300:86 , srv227956873:86 , srv416527857:86 , srv482411159:86 , srv506221545:86 , srv514634149:86 , srv731343367:86 , srv87019332:86 , srv871045212:86 , srv969394298:86 , srv1222150891:87 , srv1250558867:87 , srv1406625017:87 , srv1534922016:87 , srv1828559085:87 , srv1964482910:87 , srv2024203262:87 , srv2088149140:87 , srv2110836667:87 , srv406172948:87 , srv538552992:87 , srv649638533:87 , srv699197785:87 , srv799991109:87 , srv911933899:87 , srv922772861:87 , srv961321317:87 , srv1014696923:88 , srv1064076002:88 , srv107607601:88 , srv1263107680:88 , srv1329407150:88 , srv1410826442:88 , srv1432933960:88 , srv1854799754:88 , srv1984811680:88 , srv2083727223:88 , srv351101530:88 , srv63169025:88 , srv793888831:88 , srv79715193:88 , srv995702068:88 , srv996058553:88 , srv1293170057:89 , srv1401627029:89 , srv1625743213:89 , srv264971717:89 , srv1132747502:90 , srv1981885335:90 , srv403036198:90 , srv1467541395:91 , srv755010207:91 , srv1569868387:92 , srv599908824:92 , srv660140566:92 , srv74016846:92 , srv1037232168:93 , srv1827325017:93 , srv1979415199:419 }
2023-10-05 04:53:24,868 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:24,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1269081156=19, srv996058553=99, srv599908824=77, srv1445391656=30, srv995702068=98, srv345968430=65, srv506221545=72, srv1569868387=35, srv1432364168=28, srv1250558867=17, srv1329407150=22, srv1611684327=36, srv731343367=85, srv1132747502=10, srv1964482910=48, srv1511096967=33, srv1625743213=37, srv1828559085=44, srv1534922016=34, srv1293170057=20, srv403036198=68, srv63169025=78, srv871045212=93, srv406172948=69, srv1854799754=45, srv704720413=83, srv1014696923=1, srv227956873=61, srv110721899=7, srv538552992=75, srv2035090562=54, srv87019332=92, srv217935300=60, srv922772861=95, srv1432933960=29, srv1206406752=13, srv699197785=82, srv1037232168=2, srv2088149140=57, srv1124742268=9, srv961321317=96, srv1763733514=40, srv1827325017=43, srv78420693=88, srv1243644938=16, srv138687389=23, srv482411159=71, srv2054649602=55, srv180464267=41, srv1406625017=25, srv1222150891=14, srv2083727223=56, srv1467541395=32, srv1263107680=18, srv2024203262=53, srv74016846=86, srv107607601=4, srv514634149=73, srv1401627029=24, srv799991109=91, srv124289088=15, srv1979415199=49, srv1981885335=50, srv532666590=74, srv660140566=81, srv32880409=64, srv111382681=8, srv79715193=90, srv911933899=94, srv416527857=70, srv555647052=76, srv1984811680=51, srv264971717=62, srv2089907916=58, srv649638533=80, srv115159320=11, srv1708277138=39, srv1064076002=3, srv706393907=84, srv1414009685=27, srv319378747=63, srv1091777065=6, srv645199608=79, srv1874429614=46, srv2110836667=59, srv2000396980=52, srv755010207=87, srv969394298=97, srv1410826442=26, srv793888831=89, srv1825375263=42, srv1296015349=21, srv1450327863=31, srv351101530=66, srv1001707305=0, srv167011581=38, srv369734436=67, srv1181617906=12, srv1084268065=5, srv190310081=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:24,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:24,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:24,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:24,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:24,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:24,887 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:24,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:24,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:24,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:24,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:24,888 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:24,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:24,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:24,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:24,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:24,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:24,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:24,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-05 04:53:24,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-05 04:53:24,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-05 04:53:24,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-05 04:53:24,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-05 04:53:24,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-05 04:53:24,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-05 04:53:25,050 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03701918761806819, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05783214225575611, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03701918761806819, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16966666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:25,050 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-05 04:53:25,051 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17501722231578284, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03701918761806819, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05783214225575611, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03701918761806819, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16966666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2425, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-05 04:53:25,425 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 507 ms to try 5000 different iterations.  Found a solution that moves 2580 regions; Going from a computed imbalance of 0.17501722231578284 to a new imbalance of 0.016088773858400338. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022403302626502083); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03453393392871123, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2866666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022403302626502083); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04641666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:25,438 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv599908824:78 , srv1037232168:79 , srv755010207:79 , srv1981885335:80 , srv1467541395:81 , srv1625743213:81 , srv1250558867:82 , srv351101530:82 , srv403036198:82 , srv660140566:82 , srv74016846:82 , srv1132747502:83 , srv1263107680:83 , srv1569868387:83 , srv1827325017:83 , srv1828559085:83 , srv1854799754:83 , srv1984811680:83 , srv2024203262:83 , srv416527857:83 , srv538552992:83 , srv79715193:83 , srv922772861:83 , srv1014696923:84 , srv1064076002:84 , srv1293170057:84 , srv1329407150:84 , srv1534922016:84 , srv1964482910:84 , srv2083727223:84 , srv482411159:84 , srv63169025:84 , srv649638533:84 , srv699197785:84 , srv793888831:84 , srv799991109:84 , srv961321317:84 , srv995702068:84 , srv996058553:84 , srv107607601:85 , srv1222150891:85 , srv1763733514:85 , srv2035090562:85 , srv2110836667:85 , srv217935300:85 , srv32880409:85 , srv506221545:85 , srv514634149:85 , srv645199608:85 , srv731343367:85 , srv1084268065:86 , srv1091777065:86 , srv110721899:86 , srv111382681:86 , srv1124742268:86 , srv1401627029:86 , srv1410826442:86 , srv1825375263:86 , srv2000396980:86 , srv2054649602:86 , srv227956873:86 , srv264971717:86 , srv406172948:86 , srv706393907:86 , srv871045212:86 , srv911933899:86 , srv1181617906:87 , srv124289088:87 , srv1269081156:87 , srv138687389:87 , srv1406625017:87 , srv1432364168:87 , srv1432933960:87 , srv1445391656:87 , srv1450327863:87 , srv1511096967:87 , srv167011581:87 , srv2088149140:87 , srv532666590:87 , srv555647052:87 , srv704720413:87 , srv78420693:87 , srv87019332:87 , srv1001707305:88 , srv1206406752:88 , srv1611684327:88 , srv180464267:88 , srv969394298:88 , srv1296015349:89 , srv1414009685:89 , srv1708277138:89 , srv1874429614:89 , srv319378747:89 , srv345968430:89 , srv1243644938:90 , srv2089907916:90 , srv115159320:91 , srv190310081:91 , srv369734436:92 , srv1979415199:554 }
2023-10-05 04:53:25,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:25,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1269081156=19, srv996058553=99, srv599908824=77, srv1445391656=30, srv995702068=98, srv345968430=65, srv506221545=72, srv1569868387=35, srv1432364168=28, srv1250558867=17, srv1329407150=22, srv1611684327=36, srv731343367=85, srv1132747502=10, srv1964482910=48, srv1511096967=33, srv1625743213=37, srv1828559085=44, srv1534922016=34, srv1293170057=20, srv403036198=68, srv63169025=78, srv871045212=93, srv406172948=69, srv1854799754=45, srv704720413=83, srv1014696923=1, srv227956873=61, srv110721899=7, srv538552992=75, srv2035090562=54, srv87019332=92, srv217935300=60, srv922772861=95, srv1432933960=29, srv1206406752=13, srv699197785=82, srv1037232168=2, srv2088149140=57, srv1124742268=9, srv961321317=96, srv1763733514=40, srv1827325017=43, srv78420693=88, srv1243644938=16, srv138687389=23, srv482411159=71, srv2054649602=55, srv180464267=41, srv1406625017=25, srv1222150891=14, srv2083727223=56, srv1467541395=32, srv1263107680=18, srv2024203262=53, srv74016846=86, srv107607601=4, srv514634149=73, srv1401627029=24, srv799991109=91, srv124289088=15, srv1979415199=49, srv1981885335=50, srv532666590=74, srv660140566=81, srv32880409=64, srv111382681=8, srv79715193=90, srv911933899=94, srv416527857=70, srv555647052=76, srv1984811680=51, srv264971717=62, srv2089907916=58, srv649638533=80, srv115159320=11, srv1708277138=39, srv1064076002=3, srv706393907=84, srv1414009685=27, srv319378747=63, srv1091777065=6, srv645199608=79, srv1874429614=46, srv2110836667=59, srv2000396980=52, srv755010207=87, srv969394298=97, srv1410826442=26, srv793888831=89, srv1825375263=42, srv1296015349=21, srv1450327863=31, srv351101530=66, srv1001707305=0, srv167011581=38, srv369734436=67, srv1181617906=12, srv1084268065=5, srv190310081=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:25,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:25,468 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:25,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:25,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:25,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-05 04:53:25,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-05 04:53:25,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-05 04:53:25,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-05 04:53:25,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-05 04:53:25,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-05 04:53:25,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-05 04:53:25,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-05 04:53:25,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-05 04:53:25,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-05 04:53:25,636 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022403302626502083); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03453393392871123, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022403302626502083); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04641666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:25,636 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-05 04:53:25,637 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01607070831227768, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022403302626502083); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03453393392871123, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022403302626502083); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04641666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-05 04:53:25,908 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 396 ms to try 5000 different iterations.  Found a solution that moves 982 regions; Going from a computed imbalance of 0.01607070831227768 to a new imbalance of 6.582622700063089E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063269209666960285); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011271568674394604); MoveCostFunction : (multiplier=7.0, imbalance=0.10911111111111112, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063269209666960285); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:25,910 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv369734436:81 , srv1401627029:82 , srv1569868387:82 , srv1827325017:82 , srv1001707305:83 , srv115159320:83 , srv1406625017:83 , srv1432933960:83 , srv1708277138:83 , srv2088149140:83 , srv264971717:83 , srv319378747:83 , srv1084268065:84 , srv1293170057:84 , srv1410826442:84 , srv1445391656:84 , srv1450327863:84 , srv1511096967:84 , srv180464267:84 , srv190310081:84 , srv2054649602:84 , srv345968430:84 , srv406172948:84 , srv74016846:84 , srv87019332:84 , srv911933899:84 , srv969394298:84 , srv1014696923:85 , srv1037232168:85 , srv1064076002:85 , srv107607601:85 , srv1132747502:85 , srv1181617906:85 , srv1206406752:85 , srv1269081156:85 , srv1296015349:85 , srv138687389:85 , srv1432364168:85 , srv1611684327:85 , srv2083727223:85 , srv2089907916:85 , srv2110836667:85 , srv217935300:85 , srv403036198:85 , srv506221545:85 , srv660140566:85 , srv704720413:85 , srv793888831:85 , srv871045212:85 , srv1091777065:86 , srv111382681:86 , srv1222150891:86 , srv1243644938:86 , srv1329407150:86 , srv1414009685:86 , srv1467541395:86 , srv1625743213:86 , srv167011581:86 , srv1825375263:86 , srv1854799754:86 , srv1984811680:86 , srv227956873:86 , srv514634149:86 , srv532666590:86 , srv63169025:86 , srv649638533:86 , srv79715193:86 , srv799991109:86 , srv961321317:86 , srv995702068:86 , srv996058553:86 , srv110721899:87 , srv1124742268:87 , srv124289088:87 , srv1263107680:87 , srv1534922016:87 , srv1763733514:87 , srv1828559085:87 , srv1874429614:87 , srv1964482910:87 , srv1981885335:87 , srv2024203262:87 , srv2035090562:87 , srv538552992:87 , srv555647052:87 , srv599908824:87 , srv699197785:87 , srv706393907:87 , srv731343367:87 , srv755010207:87 , srv922772861:87 , srv1250558867:88 , srv32880409:88 , srv351101530:88 , srv482411159:88 , srv645199608:88 , srv78420693:88 , srv2000396980:89 , srv416527857:89 , srv1979415199:541 }
2023-10-05 04:53:25,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:25,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1269081156=19, srv996058553=99, srv599908824=77, srv1445391656=30, srv995702068=98, srv345968430=65, srv506221545=72, srv1569868387=35, srv1432364168=28, srv1250558867=17, srv1329407150=22, srv1611684327=36, srv731343367=85, srv1132747502=10, srv1964482910=48, srv1511096967=33, srv1625743213=37, srv1828559085=44, srv1534922016=34, srv1293170057=20, srv403036198=68, srv63169025=78, srv871045212=93, srv406172948=69, srv1854799754=45, srv704720413=83, srv1014696923=1, srv227956873=61, srv110721899=7, srv538552992=75, srv2035090562=54, srv87019332=92, srv217935300=60, srv922772861=95, srv1432933960=29, srv1206406752=13, srv699197785=82, srv1037232168=2, srv2088149140=57, srv1124742268=9, srv961321317=96, srv1763733514=40, srv1827325017=43, srv78420693=88, srv1243644938=16, srv138687389=23, srv482411159=71, srv2054649602=55, srv180464267=41, srv1406625017=25, srv1222150891=14, srv2083727223=56, srv1467541395=32, srv1263107680=18, srv2024203262=53, srv74016846=86, srv107607601=4, srv514634149=73, srv1401627029=24, srv799991109=91, srv124289088=15, srv1979415199=49, srv1981885335=50, srv532666590=74, srv660140566=81, srv32880409=64, srv111382681=8, srv79715193=90, srv911933899=94, srv416527857=70, srv555647052=76, srv1984811680=51, srv264971717=62, srv2089907916=58, srv649638533=80, srv115159320=11, srv1708277138=39, srv1064076002=3, srv706393907=84, srv1414009685=27, srv319378747=63, srv1091777065=6, srv645199608=79, srv1874429614=46, srv2110836667=59, srv2000396980=52, srv755010207=87, srv969394298=97, srv1410826442=26, srv793888831=89, srv1825375263=42, srv1296015349=21, srv1450327863=31, srv351101530=66, srv1001707305=0, srv167011581=38, srv369734436=67, srv1181617906=12, srv1084268065=5, srv190310081=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:25,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:25,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-05 04:53:25,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-05 04:53:25,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-05 04:53:25,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-05 04:53:25,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-05 04:53:25,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-05 04:53:26,058 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063269209666960285); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011271568674394604); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063269209666960285); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:26,058 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-05 04:53:26,059 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.513861590402425E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0063269209666960285); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.011271568674394604); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0063269209666960285); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-05 04:53:26,462 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 518 ms to try 5000 different iterations.  Found a solution that moves 272 regions; Going from a computed imbalance of 6.513861590402425E-4 to a new imbalance of 2.906370348162357E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.030222222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:26,463 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1124742268:83 , srv124289088:83 , srv1263107680:83 , srv1414009685:83 , srv2000396980:83 , srv227956873:83 , srv645199608:83 , srv107607601:84 , srv1091777065:84 , srv111382681:84 , srv1132747502:84 , srv1181617906:84 , srv1206406752:84 , srv1222150891:84 , srv1406625017:84 , srv1410826442:84 , srv1432364168:84 , srv1432933960:84 , srv1467541395:84 , srv1534922016:84 , srv1611684327:84 , srv1708277138:84 , srv1825375263:84 , srv1827325017:84 , srv1964482910:84 , srv1981885335:84 , srv2035090562:84 , srv2089907916:84 , srv264971717:84 , srv32880409:84 , srv351101530:84 , srv369734436:84 , srv416527857:84 , srv482411159:84 , srv532666590:84 , srv555647052:84 , srv63169025:84 , srv660140566:84 , srv699197785:84 , srv706393907:84 , srv731343367:84 , srv74016846:84 , srv969394298:84 , srv995702068:84 , srv996058553:84 , srv1001707305:85 , srv1014696923:85 , srv1037232168:85 , srv1064076002:85 , srv1084268065:85 , srv110721899:85 , srv115159320:85 , srv1243644938:85 , srv1250558867:85 , srv1269081156:85 , srv1293170057:85 , srv1296015349:85 , srv1329407150:85 , srv1401627029:85 , srv1450327863:85 , srv1511096967:85 , srv167011581:85 , srv1763733514:85 , srv180464267:85 , srv1828559085:85 , srv1854799754:85 , srv1874429614:85 , srv190310081:85 , srv1984811680:85 , srv2024203262:85 , srv2088149140:85 , srv2110836667:85 , srv319378747:85 , srv345968430:85 , srv406172948:85 , srv514634149:85 , srv538552992:85 , srv599908824:85 , srv649638533:85 , srv704720413:85 , srv755010207:85 , srv78420693:85 , srv793888831:85 , srv79715193:85 , srv799991109:85 , srv87019332:85 , srv871045212:85 , srv911933899:85 , srv922772861:85 , srv961321317:85 , srv138687389:86 , srv1445391656:86 , srv1569868387:86 , srv1625743213:86 , srv2054649602:86 , srv2083727223:86 , srv217935300:86 , srv403036198:86 , srv506221545:86 , srv1979415199:628 }
2023-10-05 04:53:26,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:26,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1269081156=19, srv996058553=99, srv599908824=77, srv1445391656=30, srv995702068=98, srv345968430=65, srv506221545=72, srv1569868387=35, srv1432364168=28, srv1250558867=17, srv1329407150=22, srv1611684327=36, srv731343367=85, srv1132747502=10, srv1964482910=48, srv1511096967=33, srv1625743213=37, srv1828559085=44, srv1534922016=34, srv1293170057=20, srv403036198=68, srv63169025=78, srv871045212=93, srv406172948=69, srv1854799754=45, srv704720413=83, srv1014696923=1, srv227956873=61, srv110721899=7, srv538552992=75, srv2035090562=54, srv87019332=92, srv217935300=60, srv922772861=95, srv1432933960=29, srv1206406752=13, srv699197785=82, srv1037232168=2, srv2088149140=57, srv1124742268=9, srv961321317=96, srv1763733514=40, srv1827325017=43, srv78420693=88, srv1243644938=16, srv138687389=23, srv482411159=71, srv2054649602=55, srv180464267=41, srv1406625017=25, srv1222150891=14, srv2083727223=56, srv1467541395=32, srv1263107680=18, srv2024203262=53, srv74016846=86, srv107607601=4, srv514634149=73, srv1401627029=24, srv799991109=91, srv124289088=15, srv1979415199=49, srv1981885335=50, srv532666590=74, srv660140566=81, srv32880409=64, srv111382681=8, srv79715193=90, srv911933899=94, srv416527857=70, srv555647052=76, srv1984811680=51, srv264971717=62, srv2089907916=58, srv649638533=80, srv115159320=11, srv1708277138=39, srv1064076002=3, srv706393907=84, srv1414009685=27, srv319378747=63, srv1091777065=6, srv645199608=79, srv1874429614=46, srv2110836667=59, srv2000396980=52, srv755010207=87, srv969394298=97, srv1410826442=26, srv793888831=89, srv1825375263=42, srv1296015349=21, srv1450327863=31, srv351101530=66, srv1001707305=0, srv167011581=38, srv369734436=67, srv1181617906=12, srv1084268065=5, srv190310081=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:26,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:26,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:26,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:26,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-05 04:53:26,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-05 04:53:26,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-05 04:53:26,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-05 04:53:26,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-05 04:53:26,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-05 04:53:26,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-05 04:53:26,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-05 04:53:26,624 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:26,624 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-05 04:53:26,624 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.7159118774118367E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-05 04:53:26,987 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 487 ms to try 5000 different iterations.  Found a solution that moves 24 regions; Going from a computed imbalance of 2.7159118774118367E-5 to a new imbalance of 8.427815577830097E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0026666666666666666); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:26,988 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1329407150:83 , srv138687389:83 , srv1569868387:83 , srv180464267:83 , srv2083727223:83 , srv403036198:83 , srv660140566:83 , srv78420693:83 , srv1001707305:84 , srv1014696923:84 , srv1037232168:84 , srv1064076002:84 , srv107607601:84 , srv1091777065:84 , srv110721899:84 , srv111382681:84 , srv1124742268:84 , srv1132747502:84 , srv115159320:84 , srv1181617906:84 , srv1206406752:84 , srv1222150891:84 , srv1243644938:84 , srv1250558867:84 , srv1269081156:84 , srv1293170057:84 , srv1296015349:84 , srv1401627029:84 , srv1410826442:84 , srv1432364168:84 , srv1432933960:84 , srv1445391656:84 , srv1450327863:84 , srv1467541395:84 , srv1511096967:84 , srv1534922016:84 , srv1611684327:84 , srv1625743213:84 , srv167011581:84 , srv1708277138:84 , srv1763733514:84 , srv1825375263:84 , srv1827325017:84 , srv1828559085:84 , srv1854799754:84 , srv1874429614:84 , srv190310081:84 , srv1964482910:84 , srv1984811680:84 , srv2000396980:84 , srv2024203262:84 , srv2035090562:84 , srv2054649602:84 , srv2088149140:84 , srv2089907916:84 , srv2110836667:84 , srv217935300:84 , srv264971717:84 , srv319378747:84 , srv32880409:84 , srv345968430:84 , srv351101530:84 , srv369734436:84 , srv406172948:84 , srv416527857:84 , srv482411159:84 , srv506221545:84 , srv514634149:84 , srv532666590:84 , srv538552992:84 , srv555647052:84 , srv599908824:84 , srv63169025:84 , srv649638533:84 , srv699197785:84 , srv704720413:84 , srv706393907:84 , srv731343367:84 , srv74016846:84 , srv755010207:84 , srv793888831:84 , srv79715193:84 , srv799991109:84 , srv87019332:84 , srv871045212:84 , srv911933899:84 , srv922772861:84 , srv961321317:84 , srv969394298:84 , srv995702068:84 , srv996058553:84 , srv1084268065:85 , srv124289088:85 , srv1263107680:85 , srv1406625017:85 , srv1414009685:85 , srv1981885335:85 , srv227956873:85 , srv645199608:85 , srv1979415199:684 }
2023-10-05 04:53:26,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:26,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1269081156=19, srv996058553=99, srv599908824=77, srv1445391656=30, srv995702068=98, srv345968430=65, srv506221545=72, srv1569868387=35, srv1432364168=28, srv1250558867=17, srv1329407150=22, srv1611684327=36, srv731343367=85, srv1132747502=10, srv1964482910=48, srv1511096967=33, srv1625743213=37, srv1828559085=44, srv1534922016=34, srv1293170057=20, srv403036198=68, srv63169025=78, srv871045212=93, srv406172948=69, srv1854799754=45, srv704720413=83, srv1014696923=1, srv227956873=61, srv110721899=7, srv538552992=75, srv2035090562=54, srv87019332=92, srv217935300=60, srv922772861=95, srv1432933960=29, srv1206406752=13, srv699197785=82, srv1037232168=2, srv2088149140=57, srv1124742268=9, srv961321317=96, srv1763733514=40, srv1827325017=43, srv78420693=88, srv1243644938=16, srv138687389=23, srv482411159=71, srv2054649602=55, srv180464267=41, srv1406625017=25, srv1222150891=14, srv2083727223=56, srv1467541395=32, srv1263107680=18, srv2024203262=53, srv74016846=86, srv107607601=4, srv514634149=73, srv1401627029=24, srv799991109=91, srv124289088=15, srv1979415199=49, srv1981885335=50, srv532666590=74, srv660140566=81, srv32880409=64, srv111382681=8, srv79715193=90, srv911933899=94, srv416527857=70, srv555647052=76, srv1984811680=51, srv264971717=62, srv2089907916=58, srv649638533=80, srv115159320=11, srv1708277138=39, srv1064076002=3, srv706393907=84, srv1414009685=27, srv319378747=63, srv1091777065=6, srv645199608=79, srv1874429614=46, srv2110836667=59, srv2000396980=52, srv755010207=87, srv969394298=97, srv1410826442=26, srv793888831=89, srv1825375263=42, srv1296015349=21, srv1450327863=31, srv351101530=66, srv1001707305=0, srv167011581=38, srv369734436=67, srv1181617906=12, srv1084268065=5, srv190310081=47} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:27,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:27,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-05 04:53:27,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-05 04:53:27,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-05 04:53:27,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-05 04:53:27,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-05 04:53:27,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-05 04:53:27,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-05 04:53:27,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-05 04:53:27,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-05 04:53:27,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-05 04:53:27,144 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.259763985991403E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:27,144 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1329407150:83 , srv138687389:83 , srv1569868387:83 , srv180464267:83 , srv2083727223:83 , srv403036198:83 , srv660140566:83 , srv78420693:83 , srv1001707305:84 , srv1014696923:84 , srv1037232168:84 , srv1064076002:84 , srv107607601:84 , srv1091777065:84 , srv110721899:84 , srv111382681:84 , srv1124742268:84 , srv1132747502:84 , srv115159320:84 , srv1181617906:84 , srv1206406752:84 , srv1222150891:84 , srv1243644938:84 , srv1250558867:84 , srv1269081156:84 , srv1293170057:84 , srv1296015349:84 , srv1401627029:84 , srv1410826442:84 , srv1432364168:84 , srv1432933960:84 , srv1445391656:84 , srv1450327863:84 , srv1467541395:84 , srv1511096967:84 , srv1534922016:84 , srv1611684327:84 , srv1625743213:84 , srv167011581:84 , srv1708277138:84 , srv1763733514:84 , srv1825375263:84 , srv1827325017:84 , srv1828559085:84 , srv1854799754:84 , srv1874429614:84 , srv190310081:84 , srv1964482910:84 , srv1984811680:84 , srv2000396980:84 , srv2024203262:84 , srv2035090562:84 , srv2054649602:84 , srv2088149140:84 , srv2089907916:84 , srv2110836667:84 , srv217935300:84 , srv264971717:84 , srv319378747:84 , srv32880409:84 , srv345968430:84 , srv351101530:84 , srv369734436:84 , srv406172948:84 , srv416527857:84 , srv482411159:84 , srv506221545:84 , srv514634149:84 , srv532666590:84 , srv538552992:84 , srv555647052:84 , srv599908824:84 , srv63169025:84 , srv649638533:84 , srv699197785:84 , srv704720413:84 , srv706393907:84 , srv731343367:84 , srv74016846:84 , srv755010207:84 , srv793888831:84 , srv79715193:84 , srv799991109:84 , srv87019332:84 , srv871045212:84 , srv911933899:84 , srv922772861:84 , srv961321317:84 , srv969394298:84 , srv995702068:84 , srv996058553:84 , srv1084268065:85 , srv124289088:85 , srv1263107680:85 , srv1406625017:85 , srv1414009685:85 , srv1981885335:85 , srv227956873:85 , srv645199608:85 , srv1979415199:684 }
2023-10-05 04:53:27,178 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=282 (was 282), ProcessCount=169 (was 169), AvailableMemoryMB=7337 (was 7241) - AvailableMemoryMB LEAK? -
2023-10-05 04:53:27,189 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=282, ProcessCount=169, AvailableMemoryMB=7336
2023-10-05 04:53:27,191 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-05 04:53:27,191 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-05 04:53:27,202 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1044568309:84 , srv1108813766:84 , srv1131138663:84 , srv115000810:84 , srv1187216246:84 , srv1216968620:84 , srv1231138215:84 , srv1333605557:84 , srv1398825486:84 , srv14648219:84 , srv1707436093:84 , srv1780979384:84 , srv2022658399:84 , srv2076564907:84 , srv2102435026:84 , srv437838940:84 , srv465030526:84 , srv465509482:84 , srv560543944:84 , srv606719835:84 , srv658325761:84 , srv660982764:84 , srv664450287:84 , srv748399574:84 , srv772411391:84 , srv774744460:84 , srv811322333:84 , srv923361331:84 , srv941991790:84 , srv1273314858:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-05 04:53:27,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:53:27,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1707436093=11, srv811322333=27, srv2076564907=14, srv1108813766=1, srv437838940=16, srv14648219=10, srv748399574=24, srv1216968620=5, srv115000810=3, srv1333605557=8, srv606719835=20, srv772411391=25, srv2022658399=13, srv1273314858=7, srv560543944=19, srv923361331=28, srv1131138663=2, srv2102435026=15, srv664450287=23, srv1044568309=0, srv660982764=22, srv1187216246=4, srv465509482=18, srv774744460=26, srv1398825486=9, srv658325761=21, srv941991790=29, srv465030526=17, srv1231138215=6, srv1780979384=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:53:27,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:53:27,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:53:27,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:53:27,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:53:27,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:53:27,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:53:27,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:53:27,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:53:27,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-05 04:53:27,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2095454451201118, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:53:27,238 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912853314119563, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2095454451201118, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-05 04:55:27,224 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5467040 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912853314119563 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:55:27,230 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1044568309:90 , srv1108813766:90 , srv1131138663:90 , srv115000810:90 , srv1187216246:90 , srv1216968620:90 , srv1231138215:90 , srv1273314858:90 , srv1333605557:90 , srv1398825486:90 , srv14648219:90 , srv1707436093:90 , srv1780979384:90 , srv2022658399:90 , srv2076564907:90 , srv2102435026:90 , srv437838940:90 , srv465030526:90 , srv465509482:90 , srv560543944:90 , srv606719835:90 , srv658325761:90 , srv660982764:90 , srv664450287:90 , srv748399574:90 , srv772411391:90 , srv774744460:90 , srv811322333:90 , srv923361331:90 , srv941991790:90 }
2023-10-05 04:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-05 04:55:27,231 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1707436093=11, srv811322333=27, srv2076564907=14, srv1108813766=1, srv437838940=16, srv14648219=10, srv748399574=24, srv1216968620=5, srv115000810=3, srv1333605557=8, srv606719835=20, srv772411391=25, srv2022658399=13, srv1273314858=7, srv560543944=19, srv923361331=28, srv1131138663=2, srv2102435026=15, srv664450287=23, srv1044568309=0, srv660982764=22, srv1187216246=4, srv465509482=18, srv774744460=26, srv1398825486=9, srv658325761=21, srv941991790=29, srv465030526=17, srv1231138215=6, srv1780979384=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-05 04:55:27,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-05 04:55:27,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-05 04:55:27,250 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-05 04:55:27,250 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1044568309:90 , srv1108813766:90 , srv1131138663:90 , srv115000810:90 , srv1187216246:90 , srv1216968620:90 , srv1231138215:90 , srv1273314858:90 , srv1333605557:90 , srv1398825486:90 , srv14648219:90 , srv1707436093:90 , srv1780979384:90 , srv2022658399:90 , srv2076564907:90 , srv2102435026:90 , srv437838940:90 , srv465030526:90 , srv465509482:90 , srv560543944:90 , srv606719835:90 , srv658325761:90 , srv660982764:90 , srv664450287:90 , srv748399574:90 , srv772411391:90 , srv774744460:90 , srv811322333:90 , srv923361331:90 , srv941991790:90 }
2023-10-05 04:55:27,265 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 265), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=191 (was 282), ProcessCount=169 (was 169), AvailableMemoryMB=6033 (was 7336)