2023-05-26 16:53:24,420 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-26 16:53:24,707 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-26 16:53:24,814 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-26 16:53:24,873 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-26 16:53:24,879 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-26 16:53:24,892 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=171, AvailableMemoryMB=5769
2023-05-26 16:53:24,894 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-26 16:53:24,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-26 16:53:25,022 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100650537:84 , srv1024874745:84 , srv104118182:84 , srv1081695372:84 , srv10909458:84 , srv1108872197:84 , srv1119778164:84 , srv1128488096:84 , srv1138038330:84 , srv116901086:84 , srv1180080521:84 , srv1181973055:84 , srv1195801645:84 , srv1201137660:84 , srv1212684329:84 , srv1217681392:84 , srv1221605426:84 , srv1242844408:84 , srv1293886275:84 , srv1310186485:84 , srv1342340360:84 , srv1345094297:84 , srv1353262604:84 , srv1404711332:84 , srv1414797546:84 , srv1458144899:84 , srv1470064597:84 , srv1547405991:84 , srv1548509909:84 , srv1574712777:84 , srv1599666056:84 , srv163969587:84 , srv1660146327:84 , srv1689587850:84 , srv1708643878:84 , srv171288325:84 , srv1714440323:84 , srv1735056925:84 , srv1757598369:84 , srv177474653:84 , srv1780164602:84 , srv178809769:84 , srv1796575016:84 , srv1800570868:84 , srv1839415497:84 , srv1839877323:84 , srv1862196512:84 , srv1871420512:84 , srv1891732594:84 , srv1896099257:84 , srv1910521485:84 , srv1911145951:84 , srv1951775234:84 , srv1953155003:84 , srv1979002890:84 , srv2066381375:84 , srv2084623561:84 , srv2098512167:84 , srv2127400319:84 , srv2128009933:84 , srv2142182141:84 , srv2147170555:84 , srv219281749:84 , srv249268118:84 , srv252983958:84 , srv293261315:84 , srv314325264:84 , srv362031399:84 , srv364595353:84 , srv374086942:84 , srv379742893:84 , srv388727452:84 , srv415504973:84 , srv431912753:84 , srv498916581:84 , srv501568180:84 , srv525961522:84 , srv561240353:84 , srv574825207:84 , srv580363869:84 , srv590984718:84 , srv599849927:84 , srv600410111:84 , srv619802422:84 , srv630932401:84 , srv641149177:84 , srv675289569:84 , srv695205494:84 , srv735341095:84 , srv809581109:84 , srv903674905:84 , srv92245465:84 , srv927206465:84 , srv945514024:84 , srv950365050:84 , srv964469374:84 , srv968954778:84 , srv978113393:84 , srv983051720:84 , srv517803823:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-26 16:53:25,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:53:25,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1293886275=18, srv1470064597=26, srv599849927=82, srv1911145951=51, srv1201137660=13, srv219281749=62, srv104118182=2, srv590984718=81, srv1345094297=21, srv314325264=66, srv641149177=86, srv10909458=4, srv379742893=70, srv415504973=72, srv600410111=83, srv1181973055=11, srv1599666056=30, srv178809769=41, srv1081695372=3, srv116901086=9, srv1839415497=44, srv1119778164=6, srv1796575016=42, srv1310186485=19, srv809581109=90, srv561240353=78, srv978113393=98, srv2142182141=60, srv249268118=63, srv501568180=75, srv2066381375=55, srv950365050=95, srv1180080521=10, srv1574712777=29, srv2084623561=56, srv2127400319=58, srv945514024=94, srv903674905=91, srv1024874745=1, srv1979002890=54, srv2128009933=59, srv92245465=92, srv1195801645=12, srv1871420512=47, srv2098512167=57, srv1128488096=7, srv362031399=67, srv293261315=65, srv1910521485=50, srv525961522=77, srv163969587=31, srv1138038330=8, srv983051720=99, srv2147170555=61, srv630932401=85, srv1221605426=16, srv1800570868=43, srv964469374=96, srv1735056925=37, srv1547405991=27, srv374086942=69, srv100650537=0, srv517803823=76, srv364595353=68, srv1891732594=48, srv574825207=79, srv1212684329=14, srv1414797546=24, srv171288325=35, srv1953155003=53, srv1108872197=5, srv695205494=88, srv1780164602=40, srv252983958=64, srv388727452=71, srv1548509909=28, srv1757598369=38, srv1217681392=15, srv1404711332=23, srv1951775234=52, srv580363869=80, srv177474653=39, srv431912753=73, srv619802422=84, srv1839877323=45, srv675289569=87, srv1862196512=46, srv1242844408=17, srv927206465=93, srv1458144899=25, srv1660146327=32, srv735341095=89, srv1353262604=22, srv1342340360=20, srv1714440323=36, srv1689587850=33, srv498916581=74, srv968954778=97, srv1708643878=34, srv1896099257=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:53:25,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:53:25,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:53:25,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-26 16:53:25,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-26 16:53:25,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-26 16:53:25,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-26 16:53:25,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-26 16:53:25,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-26 16:53:25,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-26 16:53:25,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-26 16:53:25,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-26 16:53:25,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-26 16:53:25,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-26 16:53:25,271 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:25,271 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-26 16:53:25,271 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-26 16:53:25,581 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 446 ms to try 5000 different iterations.  Found a solution that moves 4654 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.184007152998912. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04049543535171252, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05568872927174074, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5171111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04049543535171252, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1795, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24358333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:25,591 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2147170555:81 , srv219281749:81 , srv1660146327:82 , srv983051720:82 , srv1470064597:83 , srv1757598369:83 , srv1796575016:83 , srv1891732594:83 , srv362031399:83 , srv379742893:83 , srv388727452:83 , srv498916581:83 , srv100650537:84 , srv1119778164:84 , srv1128488096:84 , srv177474653:84 , srv1862196512:84 , srv1951775234:84 , srv2084623561:84 , srv252983958:84 , srv415504973:84 , srv525961522:84 , srv641149177:84 , srv903674905:84 , srv1201137660:85 , srv1342340360:85 , srv1414797546:85 , srv171288325:85 , srv178809769:85 , srv1953155003:85 , srv2098512167:85 , srv249268118:85 , srv314325264:85 , srv374086942:85 , srv431912753:85 , srv599849927:85 , srv927206465:85 , srv10909458:86 , srv1138038330:86 , srv1242844408:86 , srv1345094297:86 , srv1599666056:86 , srv163969587:86 , srv1708643878:86 , srv501568180:86 , srv580363869:86 , srv735341095:86 , srv964469374:86 , srv968954778:86 , srv104118182:87 , srv1108872197:87 , srv1180080521:87 , srv1181973055:87 , srv1212684329:87 , srv1293886275:87 , srv1404711332:87 , srv1574712777:87 , srv1689587850:87 , srv1714440323:87 , srv1871420512:87 , srv1896099257:87 , srv1911145951:87 , srv1979002890:87 , srv2127400319:87 , srv2142182141:87 , srv293261315:87 , srv561240353:87 , srv590984718:87 , srv809581109:87 , srv92245465:87 , srv950365050:87 , srv1195801645:88 , srv1310186485:88 , srv1458144899:88 , srv1547405991:88 , srv1548509909:88 , srv1910521485:88 , srv2128009933:88 , srv574825207:88 , srv600410111:88 , srv619802422:88 , srv675289569:88 , srv1221605426:89 , srv1735056925:89 , srv1839415497:89 , srv1839877323:89 , srv2066381375:89 , srv364595353:89 , srv945514024:89 , srv978113393:89 , srv1024874745:90 , srv116901086:90 , srv1081695372:91 , srv1353262604:91 , srv1780164602:91 , srv1800570868:91 , srv1217681392:92 , srv695205494:92 , srv630932401:93 , srv517803823:450 }
2023-05-26 16:53:25,595 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:53:25,596 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1293886275=18, srv1470064597=26, srv599849927=82, srv1911145951=51, srv1201137660=13, srv219281749=62, srv104118182=2, srv590984718=81, srv1345094297=21, srv314325264=66, srv641149177=86, srv10909458=4, srv379742893=70, srv415504973=72, srv600410111=83, srv1181973055=11, srv1599666056=30, srv178809769=41, srv1081695372=3, srv116901086=9, srv1839415497=44, srv1119778164=6, srv1796575016=42, srv1310186485=19, srv809581109=90, srv561240353=78, srv978113393=98, srv2142182141=60, srv249268118=63, srv501568180=75, srv2066381375=55, srv950365050=95, srv1180080521=10, srv1574712777=29, srv2084623561=56, srv2127400319=58, srv945514024=94, srv903674905=91, srv1024874745=1, srv1979002890=54, srv2128009933=59, srv92245465=92, srv1195801645=12, srv1871420512=47, srv2098512167=57, srv1128488096=7, srv362031399=67, srv293261315=65, srv1910521485=50, srv525961522=77, srv163969587=31, srv1138038330=8, srv983051720=99, srv2147170555=61, srv630932401=85, srv1221605426=16, srv1800570868=43, srv964469374=96, srv1735056925=37, srv1547405991=27, srv374086942=69, srv100650537=0, srv517803823=76, srv364595353=68, srv1891732594=48, srv574825207=79, srv1212684329=14, srv1414797546=24, srv171288325=35, srv1953155003=53, srv1108872197=5, srv695205494=88, srv1780164602=40, srv252983958=64, srv388727452=71, srv1548509909=28, srv1757598369=38, srv1217681392=15, srv1404711332=23, srv1951775234=52, srv580363869=80, srv177474653=39, srv431912753=73, srv619802422=84, srv1839877323=45, srv675289569=87, srv1862196512=46, srv1242844408=17, srv927206465=93, srv1458144899=25, srv1660146327=32, srv735341095=89, srv1353262604=22, srv1342340360=20, srv1714440323=36, srv1689587850=33, srv498916581=74, srv968954778=97, srv1708643878=34, srv1896099257=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:53:25,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:53:25,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-26 16:53:25,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-26 16:53:25,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-26 16:53:25,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-26 16:53:25,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-26 16:53:25,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-26 16:53:25,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-26 16:53:25,754 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04049543535171252, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05568872927174074, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04049543535171252, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1795, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24358333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:25,754 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-26 16:53:25,755 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1839745649943946, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04049543535171252, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05568872927174074, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04049543535171252, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1795, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24358333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-26 16:53:26,131 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 491 ms to try 5000 different iterations.  Found a solution that moves 2633 regions; Going from a computed imbalance of 0.1839745649943946 to a new imbalance of 0.015329506857661759. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02293524992208831); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033541437929244945, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29255555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02293524992208831); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:26,143 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv630932401:79 , srv1024874745:80 , srv1800570868:80 , srv695205494:80 , srv1353262604:81 , srv1081695372:82 , srv1217681392:82 , srv1221605426:82 , srv2066381375:82 , srv945514024:82 , srv104118182:83 , srv1108872197:83 , srv116901086:83 , srv1181973055:83 , srv1310186485:83 , srv1548509909:83 , srv1735056925:83 , srv1780164602:83 , srv1839415497:83 , srv574825207:83 , srv978113393:83 , srv1195801645:84 , srv1212684329:84 , srv1242844408:84 , srv163969587:84 , srv1689587850:84 , srv1714440323:84 , srv1839877323:84 , srv1910521485:84 , srv1979002890:84 , srv2127400319:84 , srv2128009933:84 , srv293261315:84 , srv364595353:84 , srv590984718:84 , srv600410111:84 , srv10909458:85 , srv1119778164:85 , srv1180080521:85 , srv1201137660:85 , srv1404711332:85 , srv1574712777:85 , srv1599666056:85 , srv1708643878:85 , srv1871420512:85 , srv1911145951:85 , srv2142182141:85 , srv501568180:85 , srv580363869:85 , srv675289569:85 , srv1138038330:86 , srv1345094297:86 , srv1458144899:86 , srv1547405991:86 , srv1951775234:86 , srv2098512167:86 , srv249268118:86 , srv314325264:86 , srv619802422:86 , srv735341095:86 , srv809581109:86 , srv92245465:86 , srv927206465:86 , srv964469374:86 , srv1128488096:87 , srv1293886275:87 , srv1342340360:87 , srv178809769:87 , srv1896099257:87 , srv374086942:87 , srv561240353:87 , srv599849927:87 , srv641149177:87 , srv1414797546:88 , srv171288325:88 , srv1757598369:88 , srv177474653:88 , srv1796575016:88 , srv1891732594:88 , srv1953155003:88 , srv2084623561:88 , srv431912753:88 , srv525961522:88 , srv903674905:88 , srv950365050:88 , srv1660146327:89 , srv1862196512:89 , srv252983958:89 , srv362031399:89 , srv388727452:89 , srv415504973:89 , srv498916581:89 , srv968954778:89 , srv983051720:89 , srv100650537:90 , srv379742893:90 , srv1470064597:91 , srv2147170555:92 , srv219281749:92 , srv517803823:528 }
2023-05-26 16:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:53:26,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1293886275=18, srv1470064597=26, srv599849927=82, srv1911145951=51, srv1201137660=13, srv219281749=62, srv104118182=2, srv590984718=81, srv1345094297=21, srv314325264=66, srv641149177=86, srv10909458=4, srv379742893=70, srv415504973=72, srv600410111=83, srv1181973055=11, srv1599666056=30, srv178809769=41, srv1081695372=3, srv116901086=9, srv1839415497=44, srv1119778164=6, srv1796575016=42, srv1310186485=19, srv809581109=90, srv561240353=78, srv978113393=98, srv2142182141=60, srv249268118=63, srv501568180=75, srv2066381375=55, srv950365050=95, srv1180080521=10, srv1574712777=29, srv2084623561=56, srv2127400319=58, srv945514024=94, srv903674905=91, srv1024874745=1, srv1979002890=54, srv2128009933=59, srv92245465=92, srv1195801645=12, srv1871420512=47, srv2098512167=57, srv1128488096=7, srv362031399=67, srv293261315=65, srv1910521485=50, srv525961522=77, srv163969587=31, srv1138038330=8, srv983051720=99, srv2147170555=61, srv630932401=85, srv1221605426=16, srv1800570868=43, srv964469374=96, srv1735056925=37, srv1547405991=27, srv374086942=69, srv100650537=0, srv517803823=76, srv364595353=68, srv1891732594=48, srv574825207=79, srv1212684329=14, srv1414797546=24, srv171288325=35, srv1953155003=53, srv1108872197=5, srv695205494=88, srv1780164602=40, srv252983958=64, srv388727452=71, srv1548509909=28, srv1757598369=38, srv1217681392=15, srv1404711332=23, srv1951775234=52, srv580363869=80, srv177474653=39, srv431912753=73, srv619802422=84, srv1839877323=45, srv675289569=87, srv1862196512=46, srv1242844408=17, srv927206465=93, srv1458144899=25, srv1660146327=32, srv735341095=89, srv1353262604=22, srv1342340360=20, srv1714440323=36, srv1689587850=33, srv498916581=74, srv968954778=97, srv1708643878=34, srv1896099257=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:53:26,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:53:26,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:53:26,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:53:26,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:53:26,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:53:26,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-26 16:53:26,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-26 16:53:26,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-26 16:53:26,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-26 16:53:26,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-26 16:53:26,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-26 16:53:26,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-26 16:53:26,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-26 16:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-26 16:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-26 16:53:26,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-26 16:53:26,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-26 16:53:26,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-26 16:53:26,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-26 16:53:26,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02293524992208831); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033541437929244945, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02293524992208831); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:26,321 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-26 16:53:26,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015311070197607125, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02293524992208831); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.033541437929244945, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02293524992208831); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-26 16:53:26,621 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 409 ms to try 5000 different iterations.  Found a solution that moves 1003 regions; Going from a computed imbalance of 0.015311070197607125 to a new imbalance of 3.119983390893192E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00498657631892483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007415820138576829); MoveCostFunction : (multiplier=7.0, imbalance=0.11144444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00498657631892483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00275); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:26,627 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv968954778:82 , srv100650537:83 , srv1217681392:83 , srv1547405991:83 , srv619802422:83 , srv950365050:83 , srv1081695372:84 , srv116901086:84 , srv1293886275:84 , srv1414797546:84 , srv1458144899:84 , srv1470064597:84 , srv171288325:84 , srv1780164602:84 , srv1896099257:84 , srv219281749:84 , srv252983958:84 , srv364595353:84 , srv415504973:84 , srv431912753:84 , srv561240353:84 , srv809581109:84 , srv1138038330:85 , srv1180080521:85 , srv1195801645:85 , srv1342340360:85 , srv1345094297:85 , srv1353262604:85 , srv178809769:85 , srv1839415497:85 , srv1862196512:85 , srv1911145951:85 , srv1953155003:85 , srv2128009933:85 , srv2142182141:85 , srv2147170555:85 , srv362031399:85 , srv574825207:85 , srv599849927:85 , srv600410111:85 , srv675289569:85 , srv903674905:85 , srv978113393:85 , srv1128488096:86 , srv1212684329:86 , srv1404711332:86 , srv1574712777:86 , srv1599666056:86 , srv1714440323:86 , srv1735056925:86 , srv1757598369:86 , srv1796575016:86 , srv1839877323:86 , srv1871420512:86 , srv1891732594:86 , srv1910521485:86 , srv2066381375:86 , srv2084623561:86 , srv314325264:86 , srv374086942:86 , srv379742893:86 , srv388727452:86 , srv498916581:86 , srv525961522:86 , srv630932401:86 , srv695205494:86 , srv92245465:86 , srv964469374:86 , srv983051720:86 , srv1024874745:87 , srv104118182:87 , srv10909458:87 , srv1181973055:87 , srv1201137660:87 , srv1221605426:87 , srv1242844408:87 , srv1310186485:87 , srv1548509909:87 , srv1660146327:87 , srv1689587850:87 , srv1708643878:87 , srv177474653:87 , srv1800570868:87 , srv1951775234:87 , srv1979002890:87 , srv2098512167:87 , srv2127400319:87 , srv249268118:87 , srv293261315:87 , srv501568180:87 , srv580363869:87 , srv590984718:87 , srv641149177:87 , srv735341095:87 , srv945514024:87 , srv1108872197:88 , srv1119778164:88 , srv163969587:88 , srv927206465:88 , srv517803823:524 }
2023-05-26 16:53:26,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:53:26,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1293886275=18, srv1470064597=26, srv599849927=82, srv1911145951=51, srv1201137660=13, srv219281749=62, srv104118182=2, srv590984718=81, srv1345094297=21, srv314325264=66, srv641149177=86, srv10909458=4, srv379742893=70, srv415504973=72, srv600410111=83, srv1181973055=11, srv1599666056=30, srv178809769=41, srv1081695372=3, srv116901086=9, srv1839415497=44, srv1119778164=6, srv1796575016=42, srv1310186485=19, srv809581109=90, srv561240353=78, srv978113393=98, srv2142182141=60, srv249268118=63, srv501568180=75, srv2066381375=55, srv950365050=95, srv1180080521=10, srv1574712777=29, srv2084623561=56, srv2127400319=58, srv945514024=94, srv903674905=91, srv1024874745=1, srv1979002890=54, srv2128009933=59, srv92245465=92, srv1195801645=12, srv1871420512=47, srv2098512167=57, srv1128488096=7, srv362031399=67, srv293261315=65, srv1910521485=50, srv525961522=77, srv163969587=31, srv1138038330=8, srv983051720=99, srv2147170555=61, srv630932401=85, srv1221605426=16, srv1800570868=43, srv964469374=96, srv1735056925=37, srv1547405991=27, srv374086942=69, srv100650537=0, srv517803823=76, srv364595353=68, srv1891732594=48, srv574825207=79, srv1212684329=14, srv1414797546=24, srv171288325=35, srv1953155003=53, srv1108872197=5, srv695205494=88, srv1780164602=40, srv252983958=64, srv388727452=71, srv1548509909=28, srv1757598369=38, srv1217681392=15, srv1404711332=23, srv1951775234=52, srv580363869=80, srv177474653=39, srv431912753=73, srv619802422=84, srv1839877323=45, srv675289569=87, srv1862196512=46, srv1242844408=17, srv927206465=93, srv1458144899=25, srv1660146327=32, srv735341095=89, srv1353262604=22, srv1342340360=20, srv1714440323=36, srv1689587850=33, srv498916581=74, srv968954778=97, srv1708643878=34, srv1896099257=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:53:26,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:53:26,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-26 16:53:26,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-26 16:53:26,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-26 16:53:26,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-26 16:53:26,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-26 16:53:26,656 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-26 16:53:26,657 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-26 16:53:26,792 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00498657631892483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007415820138576829); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00498657631892483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00275); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:26,792 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-26 16:53:26,792 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.049751829803938E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00498657631892483); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007415820138576829); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00498657631892483); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00275); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-26 16:53:27,087 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 411 ms to try 5000 different iterations.  Found a solution that moves 245 regions; Going from a computed imbalance of 3.049751829803938E-4 to a new imbalance of 1.2953944370865754E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.02722222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:27,087 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv177474653:83 , srv1839877323:83 , srv379742893:83 , srv735341095:83 , srv92245465:83 , srv927206465:83 , srv100650537:84 , srv10909458:84 , srv1108872197:84 , srv1221605426:84 , srv1293886275:84 , srv1310186485:84 , srv1404711332:84 , srv1470064597:84 , srv1548509909:84 , srv1574712777:84 , srv163969587:84 , srv1660146327:84 , srv1689587850:84 , srv1708643878:84 , srv1780164602:84 , srv1800570868:84 , srv1862196512:84 , srv1871420512:84 , srv1891732594:84 , srv1896099257:84 , srv1910521485:84 , srv1951775234:84 , srv1953155003:84 , srv1979002890:84 , srv2084623561:84 , srv2098512167:84 , srv2127400319:84 , srv2142182141:84 , srv2147170555:84 , srv249268118:84 , srv293261315:84 , srv374086942:84 , srv388727452:84 , srv415504973:84 , srv498916581:84 , srv501568180:84 , srv525961522:84 , srv561240353:84 , srv580363869:84 , srv590984718:84 , srv630932401:84 , srv641149177:84 , srv675289569:84 , srv695205494:84 , srv945514024:84 , srv950365050:84 , srv964469374:84 , srv1024874745:85 , srv104118182:85 , srv1081695372:85 , srv1119778164:85 , srv1128488096:85 , srv1138038330:85 , srv116901086:85 , srv1180080521:85 , srv1181973055:85 , srv1195801645:85 , srv1201137660:85 , srv1212684329:85 , srv1217681392:85 , srv1342340360:85 , srv1345094297:85 , srv1353262604:85 , srv1414797546:85 , srv1458144899:85 , srv1547405991:85 , srv1599666056:85 , srv171288325:85 , srv1714440323:85 , srv1735056925:85 , srv1757598369:85 , srv178809769:85 , srv1839415497:85 , srv1911145951:85 , srv2066381375:85 , srv2128009933:85 , srv219281749:85 , srv252983958:85 , srv314325264:85 , srv362031399:85 , srv364595353:85 , srv431912753:85 , srv599849927:85 , srv600410111:85 , srv619802422:85 , srv809581109:85 , srv903674905:85 , srv978113393:85 , srv983051720:85 , srv1242844408:86 , srv1796575016:86 , srv574825207:86 , srv968954778:86 , srv517803823:640 }
2023-05-26 16:53:27,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:53:27,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1293886275=18, srv1470064597=26, srv599849927=82, srv1911145951=51, srv1201137660=13, srv219281749=62, srv104118182=2, srv590984718=81, srv1345094297=21, srv314325264=66, srv641149177=86, srv10909458=4, srv379742893=70, srv415504973=72, srv600410111=83, srv1181973055=11, srv1599666056=30, srv178809769=41, srv1081695372=3, srv116901086=9, srv1839415497=44, srv1119778164=6, srv1796575016=42, srv1310186485=19, srv809581109=90, srv561240353=78, srv978113393=98, srv2142182141=60, srv249268118=63, srv501568180=75, srv2066381375=55, srv950365050=95, srv1180080521=10, srv1574712777=29, srv2084623561=56, srv2127400319=58, srv945514024=94, srv903674905=91, srv1024874745=1, srv1979002890=54, srv2128009933=59, srv92245465=92, srv1195801645=12, srv1871420512=47, srv2098512167=57, srv1128488096=7, srv362031399=67, srv293261315=65, srv1910521485=50, srv525961522=77, srv163969587=31, srv1138038330=8, srv983051720=99, srv2147170555=61, srv630932401=85, srv1221605426=16, srv1800570868=43, srv964469374=96, srv1735056925=37, srv1547405991=27, srv374086942=69, srv100650537=0, srv517803823=76, srv364595353=68, srv1891732594=48, srv574825207=79, srv1212684329=14, srv1414797546=24, srv171288325=35, srv1953155003=53, srv1108872197=5, srv695205494=88, srv1780164602=40, srv252983958=64, srv388727452=71, srv1548509909=28, srv1757598369=38, srv1217681392=15, srv1404711332=23, srv1951775234=52, srv580363869=80, srv177474653=39, srv431912753=73, srv619802422=84, srv1839877323=45, srv675289569=87, srv1862196512=46, srv1242844408=17, srv927206465=93, srv1458144899=25, srv1660146327=32, srv735341095=89, srv1353262604=22, srv1342340360=20, srv1714440323=36, srv1689587850=33, srv498916581=74, srv968954778=97, srv1708643878=34, srv1896099257=49} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:53:27,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:53:27,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-26 16:53:27,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-26 16:53:27,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-26 16:53:27,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-26 16:53:27,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-26 16:53:27,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-26 16:53:27,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-26 16:53:27,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-26 16:53:27,228 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.1238417704179081E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:27,229 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv177474653:83 , srv1839877323:83 , srv379742893:83 , srv735341095:83 , srv92245465:83 , srv927206465:83 , srv100650537:84 , srv10909458:84 , srv1108872197:84 , srv1221605426:84 , srv1293886275:84 , srv1310186485:84 , srv1404711332:84 , srv1470064597:84 , srv1548509909:84 , srv1574712777:84 , srv163969587:84 , srv1660146327:84 , srv1689587850:84 , srv1708643878:84 , srv1780164602:84 , srv1800570868:84 , srv1862196512:84 , srv1871420512:84 , srv1891732594:84 , srv1896099257:84 , srv1910521485:84 , srv1951775234:84 , srv1953155003:84 , srv1979002890:84 , srv2084623561:84 , srv2098512167:84 , srv2127400319:84 , srv2142182141:84 , srv2147170555:84 , srv249268118:84 , srv293261315:84 , srv374086942:84 , srv388727452:84 , srv415504973:84 , srv498916581:84 , srv501568180:84 , srv525961522:84 , srv561240353:84 , srv580363869:84 , srv590984718:84 , srv630932401:84 , srv641149177:84 , srv675289569:84 , srv695205494:84 , srv945514024:84 , srv950365050:84 , srv964469374:84 , srv1024874745:85 , srv104118182:85 , srv1081695372:85 , srv1119778164:85 , srv1128488096:85 , srv1138038330:85 , srv116901086:85 , srv1180080521:85 , srv1181973055:85 , srv1195801645:85 , srv1201137660:85 , srv1212684329:85 , srv1217681392:85 , srv1342340360:85 , srv1345094297:85 , srv1353262604:85 , srv1414797546:85 , srv1458144899:85 , srv1547405991:85 , srv1599666056:85 , srv171288325:85 , srv1714440323:85 , srv1735056925:85 , srv1757598369:85 , srv178809769:85 , srv1839415497:85 , srv1911145951:85 , srv2066381375:85 , srv2128009933:85 , srv219281749:85 , srv252983958:85 , srv314325264:85 , srv362031399:85 , srv364595353:85 , srv431912753:85 , srv599849927:85 , srv600410111:85 , srv619802422:85 , srv809581109:85 , srv903674905:85 , srv978113393:85 , srv983051720:85 , srv1242844408:86 , srv1796575016:86 , srv574825207:86 , srv968954778:86 , srv517803823:640 }
2023-05-26 16:53:27,252 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=345 (was 345), ProcessCount=171 (was 171), AvailableMemoryMB=5260 (was 5769)
2023-05-26 16:53:27,260 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=345, ProcessCount=171, AvailableMemoryMB=5259
2023-05-26 16:53:27,260 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-26 16:53:27,261 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-26 16:53:27,273 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1041397436:84 , srv1075126373:84 , srv1118347691:84 , srv1123042361:84 , srv1155817550:84 , srv127274055:84 , srv1353003408:84 , srv1364632233:84 , srv1437739745:84 , srv148272240:84 , srv1702003806:84 , srv1894748738:84 , srv1905391483:84 , srv2010327041:84 , srv2089876074:84 , srv2116354885:84 , srv307067704:84 , srv337967177:84 , srv494992857:84 , srv539016041:84 , srv55016881:84 , srv567393118:84 , srv767730193:84 , srv805022499:84 , srv829021930:84 , srv873078351:84 , srv89442557:84 , srv925840692:84 , srv954286134:84 , srv1443235912:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-26 16:53:27,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:53:27,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv539016041=20, srv307067704=17, srv1155817550=4, srv494992857=19, srv2089876074=15, srv805022499=24, srv1353003408=6, srv1437739745=8, srv1364632233=7, srv829021930=25, srv1894748738=12, srv337967177=18, srv1041397436=0, srv925840692=28, srv873078351=26, srv1123042361=3, srv1118347691=2, srv567393118=22, srv1075126373=1, srv55016881=21, srv89442557=27, srv767730193=23, srv1443235912=9, srv1702003806=11, srv1905391483=13, srv2116354885=16, srv127274055=5, srv148272240=10, srv2010327041=14, srv954286134=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:53:27,284 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:53:27,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:53:27,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-26 16:53:27,298 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0975208131724947, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:53:27,299 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912500328162239, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0975208131724947, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-26 16:55:27,289 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5807675 different iterations.  Found a solution that moves 1875 regions; Going from a computed imbalance of 0.9912500328162239 to a new imbalance of 4.376343537466002E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:55:27,294 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1041397436:90 , srv1075126373:90 , srv1118347691:90 , srv1123042361:90 , srv1155817550:90 , srv127274055:90 , srv1353003408:90 , srv1364632233:90 , srv1437739745:90 , srv1443235912:90 , srv148272240:90 , srv1702003806:90 , srv1894748738:90 , srv1905391483:90 , srv2010327041:90 , srv2089876074:90 , srv2116354885:90 , srv307067704:90 , srv337967177:90 , srv494992857:90 , srv539016041:90 , srv55016881:90 , srv567393118:90 , srv767730193:90 , srv805022499:90 , srv829021930:90 , srv873078351:90 , srv89442557:90 , srv925840692:90 , srv954286134:90 }
2023-05-26 16:55:27,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-26 16:55:27,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv539016041=20, srv307067704=17, srv1155817550=4, srv494992857=19, srv2089876074=15, srv805022499=24, srv1353003408=6, srv1437739745=8, srv1364632233=7, srv829021930=25, srv1894748738=12, srv337967177=18, srv1041397436=0, srv925840692=28, srv873078351=26, srv1123042361=3, srv1118347691=2, srv567393118=22, srv1075126373=1, srv55016881=21, srv89442557=27, srv767730193=23, srv1443235912=9, srv1702003806=11, srv1905391483=13, srv2116354885=16, srv127274055=5, srv148272240=10, srv2010327041=14, srv954286134=29} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-26 16:55:27,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-26 16:55:27,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-26 16:55:27,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-26 16:55:27,321 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-26 16:55:27,321 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1041397436:90 , srv1075126373:90 , srv1118347691:90 , srv1123042361:90 , srv1155817550:90 , srv127274055:90 , srv1353003408:90 , srv1364632233:90 , srv1437739745:90 , srv1443235912:90 , srv148272240:90 , srv1702003806:90 , srv1894748738:90 , srv1905391483:90 , srv2010327041:90 , srv2089876074:90 , srv2116354885:90 , srv307067704:90 , srv337967177:90 , srv494992857:90 , srv539016041:90 , srv55016881:90 , srv567393118:90 , srv767730193:90 , srv805022499:90 , srv829021930:90 , srv873078351:90 , srv89442557:90 , srv925840692:90 , srv954286134:90 }
2023-05-26 16:55:27,336 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=294 (was 345), ProcessCount=171 (was 171), AvailableMemoryMB=5194 (was 5259)