2024-01-03 01:53:16,333 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-03 01:53:16,662 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-03 01:53:16,787 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-03 01:53:16,856 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-03 01:53:16,863 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-03 01:53:16,878 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=313, ProcessCount=166, AvailableMemoryMB=3814
2024-01-03 01:53:16,881 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-03 01:53:16,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-03 01:53:17,022 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1023774538:84 , srv1081310135:84 , srv1096906851:84 , srv1102245486:84 , srv1102841359:84 , srv117769772:84 , srv1305433814:84 , srv1327298257:84 , srv1375743505:84 , srv1380179554:84 , srv1383650539:84 , srv1402564816:84 , srv1420178958:84 , srv1430538694:84 , srv1436380325:84 , srv1438039115:84 , srv1438739175:84 , srv1456260697:84 , srv1462999411:84 , srv1495465038:84 , srv1519577399:84 , srv1523598063:84 , srv1538611105:84 , srv1553906880:84 , srv1558191383:84 , srv1564648396:84 , srv1568909290:84 , srv1589675679:84 , srv1610328271:84 , srv162153079:84 , srv1655124270:84 , srv1670092857:84 , srv1683317904:84 , srv1732163821:84 , srv1765814861:84 , srv1774374915:84 , srv1778749740:84 , srv1785123187:84 , srv181502783:84 , srv1877287220:84 , srv1882302855:84 , srv1910237076:84 , srv1913786096:84 , srv1915068857:84 , srv192229975:84 , srv1934922974:84 , srv1942518221:84 , srv1964339072:84 , srv201374662:84 , srv2014092231:84 , srv2035561716:84 , srv2052108427:84 , srv2054787879:84 , srv2074632580:84 , srv2105574965:84 , srv213353338:84 , srv218567068:84 , srv256062770:84 , srv319582700:84 , srv322297250:84 , srv332509825:84 , srv401779963:84 , srv416876689:84 , srv459813449:84 , srv466739550:84 , srv477173989:84 , srv487330959:84 , srv492486423:84 , srv556761017:84 , srv561541841:84 , srv592727420:84 , srv607849478:84 , srv628799769:84 , srv701154230:84 , srv706884903:84 , srv708021904:84 , srv714242844:84 , srv715779735:84 , srv721845374:84 , srv726401632:84 , srv731385358:84 , srv73718667:84 , srv744074318:84 , srv796009357:84 , srv810798231:84 , srv817392256:84 , srv832905763:84 , srv834562500:84 , srv84701742:84 , srv850805922:84 , srv875650411:84 , srv876211285:84 , srv915530877:84 , srv923819686:84 , srv966591138:84 , srv973529293:84 , srv975256311:84 , srv987477609:84 , srv999022637:84 , srv1403724804:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-03 01:53:17,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:53:17,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv561541841=70, srv923819686=94, srv1683317904=33, srv1380179554=9, srv73718667=82, srv999022637=99, srv726401632=80, srv492486423=68, srv1913786096=43, srv1375743505=8, srv1523598063=22, srv915530877=93, srv1877287220=40, srv1436380325=15, srv1403724804=12, srv556761017=69, srv1732163821=34, srv1102841359=4, srv1383650539=10, srv2105574965=55, srv1774374915=36, srv1942518221=47, srv201374662=49, srv796009357=84, srv213353338=56, srv607849478=72, srv1765814861=35, srv459813449=64, srv1778749740=37, srv2054787879=53, srv1023774538=0, srv1589675679=28, srv1915068857=44, srv817392256=86, srv973529293=96, srv466739550=65, srv875650411=91, srv192229975=45, srv1553906880=24, srv1564648396=26, srv162153079=30, srv706884903=75, srv1910237076=42, srv987477609=98, srv1568909290=27, srv1610328271=29, srv810798231=85, srv181502783=39, srv628799769=73, srv319582700=59, srv1081310135=1, srv322297250=60, srv1305433814=6, srv416876689=63, srv966591138=95, srv1882302855=41, srv1438739175=17, srv117769772=5, srv1934922974=46, srv1430538694=14, srv1420178958=13, srv1785123187=38, srv1538611105=23, srv487330959=67, srv218567068=57, srv1519577399=21, srv1402564816=11, srv401779963=62, srv744074318=83, srv2052108427=52, srv1456260697=18, srv1495465038=20, srv1964339072=48, srv1096906851=2, srv1327298257=7, srv721845374=79, srv592727420=71, srv850805922=90, srv2035561716=51, srv715779735=78, srv876211285=92, srv1438039115=16, srv332509825=61, srv714242844=77, srv708021904=76, srv1462999411=19, srv256062770=58, srv84701742=89, srv1670092857=32, srv1655124270=31, srv1558191383=25, srv2014092231=50, srv731385358=81, srv477173989=66, srv2074632580=54, srv1102245486=3, srv832905763=87, srv701154230=74, srv975256311=97, srv834562500=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:53:17,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:53:17,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:53:17,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:53:17,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:53:17,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:53:17,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:53:17,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:53:17,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:53:17,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:53:17,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:53:17,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:53:17,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:53:17,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:53:17,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:53:17,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-03 01:53:17,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-03 01:53:17,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-03 01:53:17,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-03 01:53:17,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-03 01:53:17,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-03 01:53:17,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-03 01:53:17,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-03 01:53:17,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-03 01:53:17,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-03 01:53:17,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-03 01:53:17,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-03 01:53:17,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-03 01:53:17,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-03 01:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-03 01:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-03 01:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-03 01:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-03 01:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-03 01:53:17,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-03 01:53:17,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-03 01:53:17,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-03 01:53:17,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-03 01:53:17,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-03 01:53:17,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-03 01:53:17,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-03 01:53:17,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-03 01:53:17,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-03 01:53:17,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-03 01:53:17,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-03 01:53:17,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-03 01:53:17,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-03 01:53:17,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-03 01:53:17,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-03 01:53:17,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-03 01:53:17,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-03 01:53:17,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-03 01:53:17,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-03 01:53:17,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-03 01:53:17,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-03 01:53:17,153 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-03 01:53:17,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-03 01:53:17,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-03 01:53:17,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-03 01:53:17,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-03 01:53:17,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-03 01:53:17,476 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:17,477 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-03 01:53:17,477 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-03 01:53:17,864 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 683 ms to try 5000 different iterations.  Found a solution that moves 4714 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1715771347299997. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039180404894496944, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055624200015685264, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5237777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039180404894496944, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.167, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23058333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:17,906 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1402564816:82 , srv1877287220:82 , srv701154230:82 , srv1023774538:83 , srv117769772:83 , srv1305433814:83 , srv1383650539:83 , srv1438739175:83 , srv1462999411:83 , srv162153079:83 , srv487330959:83 , srv714242844:83 , srv1102245486:84 , srv1102841359:84 , srv1589675679:84 , srv192229975:84 , srv1942518221:84 , srv2074632580:84 , srv466739550:84 , srv715779735:84 , srv726401632:84 , srv817392256:84 , srv975256311:84 , srv987477609:84 , srv999022637:84 , srv1081310135:85 , srv1420178958:85 , srv1564648396:85 , srv1683317904:85 , srv1732163821:85 , srv181502783:85 , srv2035561716:85 , srv218567068:85 , srv401779963:85 , srv810798231:85 , srv875650411:85 , srv1327298257:86 , srv1375743505:86 , srv1380179554:86 , srv1430538694:86 , srv1495465038:86 , srv1519577399:86 , srv1765814861:86 , srv1774374915:86 , srv1910237076:86 , srv1913786096:86 , srv2054787879:86 , srv2105574965:86 , srv556761017:86 , srv592727420:86 , srv607849478:86 , srv796009357:86 , srv834562500:86 , srv876211285:86 , srv923819686:86 , srv966591138:86 , srv1436380325:87 , srv1456260697:87 , srv1568909290:87 , srv1655124270:87 , srv1778749740:87 , srv1882302855:87 , srv1915068857:87 , srv1964339072:87 , srv201374662:87 , srv2014092231:87 , srv319582700:87 , srv322297250:87 , srv416876689:87 , srv492486423:87 , srv731385358:87 , srv84701742:87 , srv915530877:87 , srv1096906851:88 , srv256062770:88 , srv628799769:88 , srv706884903:88 , srv721845374:88 , srv73718667:88 , srv744074318:88 , srv832905763:88 , srv973529293:88 , srv1670092857:89 , srv1785123187:89 , srv213353338:89 , srv1523598063:90 , srv1538611105:90 , srv1558191383:90 , srv1610328271:90 , srv561541841:90 , srv708021904:90 , srv850805922:90 , srv2052108427:91 , srv1553906880:92 , srv332509825:92 , srv477173989:93 , srv1934922974:94 , srv459813449:95 , srv1438039115:97 , srv1403724804:438 }
2024-01-03 01:53:17,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:53:17,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv561541841=70, srv923819686=94, srv1683317904=33, srv1380179554=9, srv73718667=82, srv999022637=99, srv726401632=80, srv492486423=68, srv1913786096=43, srv1375743505=8, srv1523598063=22, srv915530877=93, srv1877287220=40, srv1436380325=15, srv1403724804=12, srv556761017=69, srv1732163821=34, srv1102841359=4, srv1383650539=10, srv2105574965=55, srv1774374915=36, srv1942518221=47, srv201374662=49, srv796009357=84, srv213353338=56, srv607849478=72, srv1765814861=35, srv459813449=64, srv1778749740=37, srv2054787879=53, srv1023774538=0, srv1589675679=28, srv1915068857=44, srv817392256=86, srv973529293=96, srv466739550=65, srv875650411=91, srv192229975=45, srv1553906880=24, srv1564648396=26, srv162153079=30, srv706884903=75, srv1910237076=42, srv987477609=98, srv1568909290=27, srv1610328271=29, srv810798231=85, srv181502783=39, srv628799769=73, srv319582700=59, srv1081310135=1, srv322297250=60, srv1305433814=6, srv416876689=63, srv966591138=95, srv1882302855=41, srv1438739175=17, srv117769772=5, srv1934922974=46, srv1430538694=14, srv1420178958=13, srv1785123187=38, srv1538611105=23, srv487330959=67, srv218567068=57, srv1519577399=21, srv1402564816=11, srv401779963=62, srv744074318=83, srv2052108427=52, srv1456260697=18, srv1495465038=20, srv1964339072=48, srv1096906851=2, srv1327298257=7, srv721845374=79, srv592727420=71, srv850805922=90, srv2035561716=51, srv715779735=78, srv876211285=92, srv1438039115=16, srv332509825=61, srv714242844=77, srv708021904=76, srv1462999411=19, srv256062770=58, srv84701742=89, srv1670092857=32, srv1655124270=31, srv1558191383=25, srv2014092231=50, srv731385358=81, srv477173989=66, srv2074632580=54, srv1102245486=3, srv832905763=87, srv701154230=74, srv975256311=97, srv834562500=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:53:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:53:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-03 01:53:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-03 01:53:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-03 01:53:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-03 01:53:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-03 01:53:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-03 01:53:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-03 01:53:18,095 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039180404894496944, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055624200015685264, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039180404894496944, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.167, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23058333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:18,095 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-03 01:53:18,096 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17154412659650267, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039180404894496944, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055624200015685264, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039180404894496944, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.167, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23058333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-03 01:53:18,357 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 397 ms to try 5000 different iterations.  Found a solution that moves 2573 regions; Going from a computed imbalance of 0.17154412659650267 to a new imbalance of 0.013017222344709612. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02074314560372531); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0339637358595216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2858888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02074314560372531); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042416666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:18,367 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1438039115:74 , srv477173989:76 , srv1934922974:77 , srv459813449:77 , srv332509825:78 , srv1553906880:80 , srv1558191383:80 , srv2052108427:80 , srv213353338:80 , srv1523598063:81 , srv1785123187:82 , srv256062770:82 , srv561541841:82 , srv708021904:82 , srv731385358:82 , srv915530877:82 , srv1096906851:83 , srv1436380325:83 , srv1610328271:83 , srv2105574965:83 , srv73718667:83 , srv1670092857:84 , srv1774374915:84 , srv1778749740:84 , srv201374662:84 , srv2014092231:84 , srv556761017:84 , srv721845374:84 , srv744074318:84 , srv1375743505:85 , srv1495465038:85 , srv1519577399:85 , srv1538611105:85 , srv1564648396:85 , srv1568909290:85 , srv1964339072:85 , srv416876689:85 , srv492486423:85 , srv592727420:85 , srv850805922:85 , srv966591138:85 , srv1102841359:86 , srv1327298257:86 , srv1456260697:86 , srv181502783:86 , srv1882302855:86 , srv1915068857:86 , srv319582700:86 , srv607849478:86 , srv810798231:86 , srv832905763:86 , srv834562500:86 , srv973529293:86 , srv1081310135:87 , srv117769772:87 , srv1380179554:87 , srv1420178958:87 , srv1732163821:87 , srv1765814861:87 , srv1910237076:87 , srv1913786096:87 , srv2035561716:87 , srv2054787879:87 , srv2074632580:87 , srv322297250:87 , srv401779963:87 , srv628799769:87 , srv706884903:87 , srv726401632:87 , srv796009357:87 , srv1438739175:88 , srv1589675679:88 , srv1683317904:88 , srv192229975:88 , srv218567068:88 , srv715779735:88 , srv84701742:88 , srv875650411:88 , srv923819686:88 , srv999022637:88 , srv1023774538:89 , srv1305433814:89 , srv1402564816:89 , srv162153079:89 , srv1877287220:89 , srv466739550:89 , srv975256311:89 , srv987477609:89 , srv1102245486:90 , srv1383650539:90 , srv1462999411:90 , srv487330959:90 , srv701154230:90 , srv714242844:90 , srv817392256:90 , srv876211285:90 , srv1430538694:91 , srv1942518221:91 , srv1655124270:93 , srv1403724804:520 }
2024-01-03 01:53:18,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:53:18,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv561541841=70, srv923819686=94, srv1683317904=33, srv1380179554=9, srv73718667=82, srv999022637=99, srv726401632=80, srv492486423=68, srv1913786096=43, srv1375743505=8, srv1523598063=22, srv915530877=93, srv1877287220=40, srv1436380325=15, srv1403724804=12, srv556761017=69, srv1732163821=34, srv1102841359=4, srv1383650539=10, srv2105574965=55, srv1774374915=36, srv1942518221=47, srv201374662=49, srv796009357=84, srv213353338=56, srv607849478=72, srv1765814861=35, srv459813449=64, srv1778749740=37, srv2054787879=53, srv1023774538=0, srv1589675679=28, srv1915068857=44, srv817392256=86, srv973529293=96, srv466739550=65, srv875650411=91, srv192229975=45, srv1553906880=24, srv1564648396=26, srv162153079=30, srv706884903=75, srv1910237076=42, srv987477609=98, srv1568909290=27, srv1610328271=29, srv810798231=85, srv181502783=39, srv628799769=73, srv319582700=59, srv1081310135=1, srv322297250=60, srv1305433814=6, srv416876689=63, srv966591138=95, srv1882302855=41, srv1438739175=17, srv117769772=5, srv1934922974=46, srv1430538694=14, srv1420178958=13, srv1785123187=38, srv1538611105=23, srv487330959=67, srv218567068=57, srv1519577399=21, srv1402564816=11, srv401779963=62, srv744074318=83, srv2052108427=52, srv1456260697=18, srv1495465038=20, srv1964339072=48, srv1096906851=2, srv1327298257=7, srv721845374=79, srv592727420=71, srv850805922=90, srv2035561716=51, srv715779735=78, srv876211285=92, srv1438039115=16, srv332509825=61, srv714242844=77, srv708021904=76, srv1462999411=19, srv256062770=58, srv84701742=89, srv1670092857=32, srv1655124270=31, srv1558191383=25, srv2014092231=50, srv731385358=81, srv477173989=66, srv2074632580=54, srv1102245486=3, srv832905763=87, srv701154230=74, srv975256311=97, srv834562500=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:53:18,383 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:53:18,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:53:18,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-03 01:53:18,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-03 01:53:18,387 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-03 01:53:18,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-03 01:53:18,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-03 01:53:18,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-03 01:53:18,510 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02074314560372531); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0339637358595216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02074314560372531); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042416666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:18,510 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-03 01:53:18,511 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01299920581363457, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02074314560372531); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0339637358595216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02074314560372531); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.009916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.042416666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-03 01:53:18,840 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 435 ms to try 5000 different iterations.  Found a solution that moves 996 regions; Going from a computed imbalance of 0.01299920581363457 to a new imbalance of 4.0164874956274683E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008287754140107479); MoveCostFunction : (multiplier=7.0, imbalance=0.11066666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0028333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:18,842 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1655124270:78 , srv1430538694:81 , srv850805922:82 , srv1538611105:83 , srv1910237076:83 , srv1942518221:83 , srv487330959:83 , srv628799769:83 , srv706884903:83 , srv832905763:83 , srv84701742:83 , srv876211285:83 , srv923819686:83 , srv1102245486:84 , srv1380179554:84 , srv1882302855:84 , srv2054787879:84 , srv218567068:84 , srv322297250:84 , srv466739550:84 , srv796009357:84 , srv817392256:84 , srv973529293:84 , srv987477609:84 , srv1023774538:85 , srv1081310135:85 , srv1327298257:85 , srv1383650539:85 , srv1402564816:85 , srv1420178958:85 , srv1462999411:85 , srv1670092857:85 , srv1683317904:85 , srv1913786096:85 , srv1915068857:85 , srv1964339072:85 , srv2035561716:85 , srv2074632580:85 , srv319582700:85 , srv401779963:85 , srv459813449:85 , srv492486423:85 , srv561541841:85 , srv607849478:85 , srv708021904:85 , srv714242844:85 , srv715779735:85 , srv744074318:85 , srv834562500:85 , srv875650411:85 , srv975256311:85 , srv999022637:85 , srv1096906851:86 , srv1305433814:86 , srv1375743505:86 , srv1438039115:86 , srv1438739175:86 , srv1456260697:86 , srv1495465038:86 , srv1519577399:86 , srv1523598063:86 , srv1553906880:86 , srv1568909290:86 , srv1589675679:86 , srv1610328271:86 , srv162153079:86 , srv1732163821:86 , srv1765814861:86 , srv1785123187:86 , srv1877287220:86 , srv192229975:86 , srv1934922974:86 , srv201374662:86 , srv416876689:86 , srv592727420:86 , srv701154230:86 , srv721845374:86 , srv726401632:86 , srv73718667:86 , srv1102841359:87 , srv117769772:87 , srv1436380325:87 , srv1564648396:87 , srv1774374915:87 , srv1778749740:87 , srv181502783:87 , srv2052108427:87 , srv556761017:87 , srv810798231:87 , srv966591138:87 , srv1558191383:88 , srv2014092231:88 , srv256062770:88 , srv477173989:88 , srv731385358:88 , srv2105574965:89 , srv213353338:89 , srv332509825:89 , srv915530877:89 , srv1403724804:550 }
2024-01-03 01:53:18,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:53:18,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv561541841=70, srv923819686=94, srv1683317904=33, srv1380179554=9, srv73718667=82, srv999022637=99, srv726401632=80, srv492486423=68, srv1913786096=43, srv1375743505=8, srv1523598063=22, srv915530877=93, srv1877287220=40, srv1436380325=15, srv1403724804=12, srv556761017=69, srv1732163821=34, srv1102841359=4, srv1383650539=10, srv2105574965=55, srv1774374915=36, srv1942518221=47, srv201374662=49, srv796009357=84, srv213353338=56, srv607849478=72, srv1765814861=35, srv459813449=64, srv1778749740=37, srv2054787879=53, srv1023774538=0, srv1589675679=28, srv1915068857=44, srv817392256=86, srv973529293=96, srv466739550=65, srv875650411=91, srv192229975=45, srv1553906880=24, srv1564648396=26, srv162153079=30, srv706884903=75, srv1910237076=42, srv987477609=98, srv1568909290=27, srv1610328271=29, srv810798231=85, srv181502783=39, srv628799769=73, srv319582700=59, srv1081310135=1, srv322297250=60, srv1305433814=6, srv416876689=63, srv966591138=95, srv1882302855=41, srv1438739175=17, srv117769772=5, srv1934922974=46, srv1430538694=14, srv1420178958=13, srv1785123187=38, srv1538611105=23, srv487330959=67, srv218567068=57, srv1519577399=21, srv1402564816=11, srv401779963=62, srv744074318=83, srv2052108427=52, srv1456260697=18, srv1495465038=20, srv1964339072=48, srv1096906851=2, srv1327298257=7, srv721845374=79, srv592727420=71, srv850805922=90, srv2035561716=51, srv715779735=78, srv876211285=92, srv1438039115=16, srv332509825=61, srv714242844=77, srv708021904=76, srv1462999411=19, srv256062770=58, srv84701742=89, srv1670092857=32, srv1655124270=31, srv1558191383=25, srv2014092231=50, srv731385358=81, srv477173989=66, srv2074632580=54, srv1102245486=3, srv832905763=87, srv701154230=74, srv975256311=97, srv834562500=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:53:18,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:53:18,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-03 01:53:18,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-03 01:53:18,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-03 01:53:18,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-03 01:53:18,862 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-03 01:53:18,973 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008287754140107479); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0028333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:18,973 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-03 01:53:18,974 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.9467460850144104E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005661173367839546); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008287754140107479); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005661173367839546); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0028333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-03 01:53:19,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 323 ms to try 5000 different iterations.  Found a solution that moves 228 regions; Going from a computed imbalance of 3.9467460850144104E-4 to a new imbalance of 1.8651418051925977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.025333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:19,200 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1383650539:83 , srv1610328271:83 , srv1765814861:83 , srv2014092231:83 , srv332509825:83 , srv876211285:83 , srv1102245486:84 , srv1305433814:84 , srv1375743505:84 , srv1430538694:84 , srv1436380325:84 , srv1456260697:84 , srv1462999411:84 , srv1538611105:84 , srv1553906880:84 , srv1558191383:84 , srv1564648396:84 , srv1568909290:84 , srv1589675679:84 , srv162153079:84 , srv1655124270:84 , srv1670092857:84 , srv1683317904:84 , srv1732163821:84 , srv1778749740:84 , srv181502783:84 , srv1877287220:84 , srv1913786096:84 , srv1915068857:84 , srv192229975:84 , srv1942518221:84 , srv2052108427:84 , srv2105574965:84 , srv319582700:84 , srv322297250:84 , srv416876689:84 , srv628799769:84 , srv701154230:84 , srv706884903:84 , srv714242844:84 , srv721845374:84 , srv810798231:84 , srv817392256:84 , srv834562500:84 , srv84701742:84 , srv966591138:84 , srv973529293:84 , srv975256311:84 , srv1023774538:85 , srv1081310135:85 , srv1096906851:85 , srv1102841359:85 , srv117769772:85 , srv1327298257:85 , srv1380179554:85 , srv1420178958:85 , srv1438039115:85 , srv1438739175:85 , srv1495465038:85 , srv1519577399:85 , srv1523598063:85 , srv1774374915:85 , srv1785123187:85 , srv1882302855:85 , srv1934922974:85 , srv1964339072:85 , srv201374662:85 , srv2035561716:85 , srv2054787879:85 , srv213353338:85 , srv218567068:85 , srv256062770:85 , srv401779963:85 , srv459813449:85 , srv466739550:85 , srv477173989:85 , srv492486423:85 , srv556761017:85 , srv561541841:85 , srv607849478:85 , srv708021904:85 , srv715779735:85 , srv726401632:85 , srv731385358:85 , srv73718667:85 , srv744074318:85 , srv796009357:85 , srv832905763:85 , srv850805922:85 , srv875650411:85 , srv915530877:85 , srv923819686:85 , srv987477609:85 , srv999022637:85 , srv1402564816:86 , srv1910237076:86 , srv2074632580:86 , srv487330959:86 , srv592727420:86 , srv1403724804:634 }
2024-01-03 01:53:19,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:53:19,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv561541841=70, srv923819686=94, srv1683317904=33, srv1380179554=9, srv73718667=82, srv999022637=99, srv726401632=80, srv492486423=68, srv1913786096=43, srv1375743505=8, srv1523598063=22, srv915530877=93, srv1877287220=40, srv1436380325=15, srv1403724804=12, srv556761017=69, srv1732163821=34, srv1102841359=4, srv1383650539=10, srv2105574965=55, srv1774374915=36, srv1942518221=47, srv201374662=49, srv796009357=84, srv213353338=56, srv607849478=72, srv1765814861=35, srv459813449=64, srv1778749740=37, srv2054787879=53, srv1023774538=0, srv1589675679=28, srv1915068857=44, srv817392256=86, srv973529293=96, srv466739550=65, srv875650411=91, srv192229975=45, srv1553906880=24, srv1564648396=26, srv162153079=30, srv706884903=75, srv1910237076=42, srv987477609=98, srv1568909290=27, srv1610328271=29, srv810798231=85, srv181502783=39, srv628799769=73, srv319582700=59, srv1081310135=1, srv322297250=60, srv1305433814=6, srv416876689=63, srv966591138=95, srv1882302855=41, srv1438739175=17, srv117769772=5, srv1934922974=46, srv1430538694=14, srv1420178958=13, srv1785123187=38, srv1538611105=23, srv487330959=67, srv218567068=57, srv1519577399=21, srv1402564816=11, srv401779963=62, srv744074318=83, srv2052108427=52, srv1456260697=18, srv1495465038=20, srv1964339072=48, srv1096906851=2, srv1327298257=7, srv721845374=79, srv592727420=71, srv850805922=90, srv2035561716=51, srv715779735=78, srv876211285=92, srv1438039115=16, srv332509825=61, srv714242844=77, srv708021904=76, srv1462999411=19, srv256062770=58, srv84701742=89, srv1670092857=32, srv1655124270=31, srv1558191383=25, srv2014092231=50, srv731385358=81, srv477173989=66, srv2074632580=54, srv1102245486=3, srv832905763=87, srv701154230=74, srv975256311=97, srv834562500=88} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:53:19,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-03 01:53:19,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-03 01:53:19,217 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-03 01:53:19,218 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-03 01:53:19,219 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-03 01:53:19,220 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-03 01:53:19,221 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-03 01:53:19,327 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7054927929458383E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:19,327 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1383650539:83 , srv1610328271:83 , srv1765814861:83 , srv2014092231:83 , srv332509825:83 , srv876211285:83 , srv1102245486:84 , srv1305433814:84 , srv1375743505:84 , srv1430538694:84 , srv1436380325:84 , srv1456260697:84 , srv1462999411:84 , srv1538611105:84 , srv1553906880:84 , srv1558191383:84 , srv1564648396:84 , srv1568909290:84 , srv1589675679:84 , srv162153079:84 , srv1655124270:84 , srv1670092857:84 , srv1683317904:84 , srv1732163821:84 , srv1778749740:84 , srv181502783:84 , srv1877287220:84 , srv1913786096:84 , srv1915068857:84 , srv192229975:84 , srv1942518221:84 , srv2052108427:84 , srv2105574965:84 , srv319582700:84 , srv322297250:84 , srv416876689:84 , srv628799769:84 , srv701154230:84 , srv706884903:84 , srv714242844:84 , srv721845374:84 , srv810798231:84 , srv817392256:84 , srv834562500:84 , srv84701742:84 , srv966591138:84 , srv973529293:84 , srv975256311:84 , srv1023774538:85 , srv1081310135:85 , srv1096906851:85 , srv1102841359:85 , srv117769772:85 , srv1327298257:85 , srv1380179554:85 , srv1420178958:85 , srv1438039115:85 , srv1438739175:85 , srv1495465038:85 , srv1519577399:85 , srv1523598063:85 , srv1774374915:85 , srv1785123187:85 , srv1882302855:85 , srv1934922974:85 , srv1964339072:85 , srv201374662:85 , srv2035561716:85 , srv2054787879:85 , srv213353338:85 , srv218567068:85 , srv256062770:85 , srv401779963:85 , srv459813449:85 , srv466739550:85 , srv477173989:85 , srv492486423:85 , srv556761017:85 , srv561541841:85 , srv607849478:85 , srv708021904:85 , srv715779735:85 , srv726401632:85 , srv731385358:85 , srv73718667:85 , srv744074318:85 , srv796009357:85 , srv832905763:85 , srv850805922:85 , srv875650411:85 , srv915530877:85 , srv923819686:85 , srv987477609:85 , srv999022637:85 , srv1402564816:86 , srv1910237076:86 , srv2074632580:86 , srv487330959:86 , srv592727420:86 , srv1403724804:634 }
2024-01-03 01:53:19,356 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=312 (was 313), ProcessCount=166 (was 166), AvailableMemoryMB=3229 (was 3814)
2024-01-03 01:53:19,363 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=312, ProcessCount=166, AvailableMemoryMB=3228
2024-01-03 01:53:19,364 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-03 01:53:19,364 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-03 01:53:19,374 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1017723236:84 , srv1057444252:84 , srv1114250188:84 , srv1245171923:84 , srv1306186773:84 , srv1306884850:84 , srv1337561711:84 , srv1391776863:84 , srv1393020721:84 , srv1418474634:84 , srv1437093947:84 , srv1709697403:84 , srv1789175134:84 , srv2033682885:84 , srv2062710343:84 , srv2106336962:84 , srv357351105:84 , srv403760314:84 , srv524739534:84 , srv528734746:84 , srv539351291:84 , srv59666674:84 , srv680549010:84 , srv735310910:84 , srv75388829:84 , srv801476414:84 , srv824795440:84 , srv853958872:84 , srv929119117:84 , srv2124336924:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-03 01:53:19,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:53:19,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv403760314=18, srv2033682885=13, srv735310910=24, srv1337561711=6, srv1306186773=4, srv1114250188=2, srv528734746=20, srv853958872=28, srv1709697403=11, srv539351291=21, srv1418474634=9, srv2106336962=15, srv1017723236=0, srv1391776863=7, srv524739534=19, srv1057444252=1, srv1245171923=3, srv801476414=26, srv2124336924=16, srv929119117=29, srv2062710343=14, srv1306884850=5, srv1789175134=12, srv1393020721=8, srv824795440=27, srv59666674=22, srv680549010=23, srv357351105=17, srv75388829=25, srv1437093947=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:53:19,384 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:53:19,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:53:19,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-03 01:53:19,405 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0984003648525507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:53:19,406 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912503099600862, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0984003648525507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-03 01:55:19,389 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6059660 different iterations.  Found a solution that moves 1857 regions; Going from a computed imbalance of 0.9912503099600862 to a new imbalance of 4.334330639506329E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:55:19,399 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1017723236:90 , srv1057444252:90 , srv1114250188:90 , srv1245171923:90 , srv1306186773:90 , srv1306884850:90 , srv1337561711:90 , srv1391776863:90 , srv1393020721:90 , srv1418474634:90 , srv1437093947:90 , srv1709697403:90 , srv1789175134:90 , srv2033682885:90 , srv2062710343:90 , srv2106336962:90 , srv2124336924:90 , srv357351105:90 , srv403760314:90 , srv524739534:90 , srv528734746:90 , srv539351291:90 , srv59666674:90 , srv680549010:90 , srv735310910:90 , srv75388829:90 , srv801476414:90 , srv824795440:90 , srv853958872:90 , srv929119117:90 }
2024-01-03 01:55:19,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-03 01:55:19,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv403760314=18, srv2033682885=13, srv735310910=24, srv1337561711=6, srv1306186773=4, srv1114250188=2, srv528734746=20, srv853958872=28, srv1709697403=11, srv539351291=21, srv1418474634=9, srv2106336962=15, srv1017723236=0, srv1391776863=7, srv524739534=19, srv1057444252=1, srv1245171923=3, srv801476414=26, srv2124336924=16, srv929119117=29, srv2062710343=14, srv1306884850=5, srv1789175134=12, srv1393020721=8, srv824795440=27, srv59666674=22, srv680549010=23, srv357351105=17, srv75388829=25, srv1437093947=10} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-03 01:55:19,405 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-03 01:55:19,406 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-03 01:55:19,407 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-03 01:55:19,424 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-03 01:55:19,425 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1017723236:90 , srv1057444252:90 , srv1114250188:90 , srv1245171923:90 , srv1306186773:90 , srv1306884850:90 , srv1337561711:90 , srv1391776863:90 , srv1393020721:90 , srv1418474634:90 , srv1437093947:90 , srv1709697403:90 , srv1789175134:90 , srv2033682885:90 , srv2062710343:90 , srv2106336962:90 , srv2124336924:90 , srv357351105:90 , srv403760314:90 , srv524739534:90 , srv528734746:90 , srv539351291:90 , srv59666674:90 , srv680549010:90 , srv735310910:90 , srv75388829:90 , srv801476414:90 , srv824795440:90 , srv853958872:90 , srv929119117:90 }
2024-01-03 01:55:19,441 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=247 (was 312), ProcessCount=167 (was 166) - ProcessCount LEAK? -, AvailableMemoryMB=3327 (was 3228) - AvailableMemoryMB LEAK? -