2023-03-23 08:56:06,731 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-23 08:56:07,039 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-23 08:56:07,144 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-23 08:56:07,200 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 08:56:07,207 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-23 08:56:07,220 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=303, ProcessCount=169, AvailableMemoryMB=6559
2023-03-23 08:56:07,222 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 08:56:07,222 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-23 08:56:07,337 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1010394490:84 , srv1027512460:84 , srv1039392332:84 , srv1062402423:84 , srv1064502566:84 , srv1087900447:84 , srv109328012:84 , srv1162651654:84 , srv1272933730:84 , srv1298972806:84 , srv132656897:84 , srv1335512559:84 , srv1343340583:84 , srv1359418044:84 , srv1386186356:84 , srv1386828976:84 , srv1418065596:84 , srv1455408605:84 , srv1462420104:84 , srv1471525293:84 , srv1502647285:84 , srv1519379194:84 , srv1536744005:84 , srv1542987652:84 , srv1591954769:84 , srv1617535518:84 , srv1655817117:84 , srv1663439608:84 , srv1675631891:84 , srv1692635455:84 , srv1718988663:84 , srv1744581560:84 , srv1802463753:84 , srv1810731201:84 , srv1816100852:84 , srv1820968868:84 , srv1833627106:84 , srv1841886051:84 , srv1859431326:84 , srv1881060607:84 , srv1885054245:84 , srv1899033489:84 , srv1900292005:84 , srv1964811004:84 , srv1966212184:84 , srv1989760201:84 , srv2042349116:84 , srv2056978732:84 , srv2067351549:84 , srv2074441794:84 , srv2096901958:84 , srv2131692509:84 , srv2134234089:84 , srv2141753053:84 , srv234653344:84 , srv238131281:84 , srv264607481:84 , srv350020447:84 , srv352694159:84 , srv361611008:84 , srv362276155:84 , srv366296420:84 , srv367191275:84 , srv388318647:84 , srv394960625:84 , srv425987317:84 , srv442335602:84 , srv442736921:84 , srv501170801:84 , srv536893167:84 , srv554178494:84 , srv555559515:84 , srv586640970:84 , srv612667166:84 , srv613805597:84 , srv629436261:84 , srv650281721:84 , srv660402926:84 , srv669897945:84 , srv694444285:84 , srv718642317:84 , srv757571264:84 , srv759649079:84 , srv787825381:84 , srv797137943:84 , srv803811853:84 , srv821919128:84 , srv838472890:84 , srv841153269:84 , srv886849065:84 , srv893877218:84 , srv898100257:84 , srv911716242:84 , srv925116579:84 , srv93962040:84 , srv943491429:84 , srv950302998:84 , srv960839325:84 , srv96827496:84 , srv224016293:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-23 08:56:07,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:56:07,353 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1802463753=32, srv1027512460=1, srv1591954769=24, srv2131692509=51, srv821919128=87, srv1841886051=37, srv352694159=59, srv394960625=65, srv1859431326=38, srv1359418044=13, srv2067351549=48, srv1718988663=30, srv96827496=99, srv501170801=69, srv238131281=56, srv1386186356=14, srv838472890=88, srv1335512559=11, srv718642317=81, srv1964811004=43, srv2056978732=47, srv362276155=61, srv1462420104=18, srv93962040=95, srv1519379194=21, srv1386828976=15, srv613805597=75, srv1064502566=4, srv950302998=97, srv629436261=76, srv132656897=10, srv1502647285=20, srv893877218=91, srv757571264=82, srv1471525293=19, srv943491429=96, srv886849065=90, srv612667166=74, srv586640970=73, srv960839325=98, srv1816100852=34, srv1655817117=26, srv554178494=71, srv1900292005=42, srv1692635455=29, srv2096901958=50, srv1542987652=23, srv224016293=54, srv669897945=79, srv234653344=55, srv1675631891=28, srv1966212184=44, srv536893167=70, srv841153269=89, srv803811853=86, srv759649079=83, srv1536744005=22, srv898100257=92, srv1298972806=9, srv1343340583=12, srv650281721=77, srv797137943=85, srv1810731201=33, srv361611008=60, srv1744581560=31, srv1820968868=35, srv1881060607=39, srv1989760201=45, srv1272933730=8, srv366296420=62, srv1039392332=2, srv787825381=84, srv1062402423=3, srv350020447=58, srv264607481=57, srv109328012=6, srv1617535518=25, srv2074441794=49, srv660402926=78, srv2141753053=53, srv425987317=66, srv1899033489=41, srv1418065596=16, srv1087900447=5, srv2042349116=46, srv442335602=67, srv911716242=93, srv1663439608=27, srv1885054245=40, srv1455408605=17, srv442736921=68, srv388318647=64, srv367191275=63, srv925116579=94, srv694444285=80, srv555559515=72, srv1833627106=36, srv2134234089=52, srv1010394490=0, srv1162651654=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:56:07,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:56:07,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:56:07,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:56:07,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:56:07,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:56:07,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:56:07,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 08:56:07,445 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 08:56:07,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 08:56:07,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 08:56:07,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 08:56:07,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 08:56:07,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 08:56:07,446 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 08:56:07,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 08:56:07,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 08:56:07,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 08:56:07,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 08:56:07,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 08:56:07,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 08:56:07,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 08:56:07,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 08:56:07,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 08:56:07,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 08:56:07,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 08:56:07,656 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:07,656 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 08:56:07,657 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 08:56:07,951 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 4663 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18133932976208833. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.043417139599171393, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06154756904029534, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5181111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.043417139599171393, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24266666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:07,969 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv132656897:81 , srv1802463753:81 , srv2134234089:81 , srv1062402423:82 , srv1298972806:82 , srv394960625:82 , srv442736921:82 , srv554178494:82 , srv787825381:82 , srv1039392332:83 , srv1455408605:83 , srv1542987652:83 , srv1841886051:83 , srv1900292005:83 , srv350020447:83 , srv425987317:83 , srv613805597:83 , srv629436261:83 , srv838472890:83 , srv109328012:84 , srv1162651654:84 , srv1663439608:84 , srv1885054245:84 , srv361611008:84 , srv759649079:84 , srv898100257:84 , srv1343340583:85 , srv1692635455:85 , srv2056978732:85 , srv2096901958:85 , srv586640970:85 , srv612667166:85 , srv650281721:85 , srv660402926:85 , srv669897945:85 , srv718642317:85 , srv797137943:85 , srv841153269:85 , srv911716242:85 , srv925116579:85 , srv950302998:85 , srv1087900447:86 , srv1418065596:86 , srv1462420104:86 , srv1519379194:86 , srv1675631891:86 , srv1966212184:86 , srv1989760201:86 , srv2141753053:86 , srv366296420:86 , srv388318647:86 , srv501170801:86 , srv536893167:86 , srv893877218:86 , srv93962040:86 , srv1010394490:87 , srv1386828976:87 , srv1471525293:87 , srv1591954769:87 , srv1655817117:87 , srv1820968868:87 , srv1859431326:87 , srv2067351549:87 , srv2074441794:87 , srv264607481:87 , srv352694159:87 , srv362276155:87 , srv367191275:87 , srv757571264:87 , srv821919128:87 , srv1064502566:88 , srv1335512559:88 , srv1386186356:88 , srv1536744005:88 , srv1617535518:88 , srv234653344:88 , srv238131281:88 , srv694444285:88 , srv803811853:88 , srv886849065:88 , srv960839325:88 , srv1359418044:89 , srv1502647285:89 , srv1881060607:89 , srv1899033489:89 , srv1964811004:89 , srv2042349116:89 , srv2131692509:89 , srv943491429:89 , srv1272933730:90 , srv1718988663:90 , srv1810731201:90 , srv1816100852:90 , srv1833627106:90 , srv1744581560:91 , srv442335602:91 , srv555559515:91 , srv1027512460:92 , srv96827496:92 , srv224016293:476 }
2023-03-23 08:56:07,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:56:07,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1802463753=32, srv1027512460=1, srv1591954769=24, srv2131692509=51, srv821919128=87, srv1841886051=37, srv352694159=59, srv394960625=65, srv1859431326=38, srv1359418044=13, srv2067351549=48, srv1718988663=30, srv96827496=99, srv501170801=69, srv238131281=56, srv1386186356=14, srv838472890=88, srv1335512559=11, srv718642317=81, srv1964811004=43, srv2056978732=47, srv362276155=61, srv1462420104=18, srv93962040=95, srv1519379194=21, srv1386828976=15, srv613805597=75, srv1064502566=4, srv950302998=97, srv629436261=76, srv132656897=10, srv1502647285=20, srv893877218=91, srv757571264=82, srv1471525293=19, srv943491429=96, srv886849065=90, srv612667166=74, srv586640970=73, srv960839325=98, srv1816100852=34, srv1655817117=26, srv554178494=71, srv1900292005=42, srv1692635455=29, srv2096901958=50, srv1542987652=23, srv224016293=54, srv669897945=79, srv234653344=55, srv1675631891=28, srv1966212184=44, srv536893167=70, srv841153269=89, srv803811853=86, srv759649079=83, srv1536744005=22, srv898100257=92, srv1298972806=9, srv1343340583=12, srv650281721=77, srv797137943=85, srv1810731201=33, srv361611008=60, srv1744581560=31, srv1820968868=35, srv1881060607=39, srv1989760201=45, srv1272933730=8, srv366296420=62, srv1039392332=2, srv787825381=84, srv1062402423=3, srv350020447=58, srv264607481=57, srv109328012=6, srv1617535518=25, srv2074441794=49, srv660402926=78, srv2141753053=53, srv425987317=66, srv1899033489=41, srv1418065596=16, srv1087900447=5, srv2042349116=46, srv442335602=67, srv911716242=93, srv1663439608=27, srv1885054245=40, srv1455408605=17, srv442736921=68, srv388318647=64, srv367191275=63, srv925116579=94, srv694444285=80, srv555559515=72, srv1833627106=36, srv2134234089=52, srv1010394490=0, srv1162651654=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:56:08,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:56:08,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:56:08,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 08:56:08,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 08:56:08,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 08:56:08,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 08:56:08,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 08:56:08,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 08:56:08,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 08:56:08,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 08:56:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 08:56:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 08:56:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 08:56:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 08:56:08,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 08:56:08,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.043417139599171393, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06154756904029534, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.043417139599171393, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24266666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:08,170 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 08:56:08,170 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.181306678738224, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.043417139599171393, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06154756904029534, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.043417139599171393, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17658333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24266666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 08:56:08,420 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 390 ms to try 5000 different iterations.  Found a solution that moves 2675 regions; Going from a computed imbalance of 0.181306678738224 to a new imbalance of 0.015350257080176101. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021167448015012646); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03164406423560035, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2972222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021167448015012646); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:08,431 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1027512460:80 , srv96827496:80 , srv1718988663:81 , srv1899033489:81 , srv1272933730:82 , srv1386828976:82 , srv1816100852:82 , srv2042349116:82 , srv238131281:82 , srv886849065:82 , srv960839325:82 , srv1964811004:83 , srv2131692509:83 , srv555559515:83 , srv694444285:83 , srv803811853:83 , srv1386186356:84 , srv1471525293:84 , srv1591954769:84 , srv1744581560:84 , srv1820968868:84 , srv1833627106:84 , srv1859431326:84 , srv2067351549:84 , srv362276155:84 , srv442335602:84 , srv1010394490:85 , srv1064502566:85 , srv1335512559:85 , srv1359418044:85 , srv1519379194:85 , srv1536744005:85 , srv1617535518:85 , srv1881060607:85 , srv1989760201:85 , srv2074441794:85 , srv234653344:85 , srv352694159:85 , srv501170801:85 , srv586640970:85 , srv757571264:85 , srv925116579:85 , srv943491429:85 , srv950302998:85 , srv1162651654:86 , srv1502647285:86 , srv1655817117:86 , srv1692635455:86 , srv1810731201:86 , srv2096901958:86 , srv2141753053:86 , srv367191275:86 , srv388318647:86 , srv612667166:86 , srv660402926:86 , srv893877218:86 , srv93962040:86 , srv1039392332:87 , srv1087900447:87 , srv109328012:87 , srv1343340583:87 , srv1418065596:87 , srv1462420104:87 , srv1675631891:87 , srv2056978732:87 , srv264607481:87 , srv425987317:87 , srv650281721:87 , srv669897945:87 , srv718642317:87 , srv132656897:88 , srv366296420:88 , srv394960625:88 , srv442736921:88 , srv759649079:88 , srv797137943:88 , srv821919128:88 , srv838472890:88 , srv898100257:88 , srv911716242:88 , srv1062402423:89 , srv1455408605:89 , srv1542987652:89 , srv1885054245:89 , srv1966212184:89 , srv2134234089:89 , srv350020447:89 , srv361611008:89 , srv536893167:89 , srv554178494:89 , srv613805597:89 , srv841153269:89 , srv1663439608:90 , srv1900292005:90 , srv787825381:90 , srv1802463753:91 , srv1841886051:91 , srv629436261:91 , srv1298972806:93 , srv224016293:486 }
2023-03-23 08:56:08,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:56:08,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1802463753=32, srv1027512460=1, srv1591954769=24, srv2131692509=51, srv821919128=87, srv1841886051=37, srv352694159=59, srv394960625=65, srv1859431326=38, srv1359418044=13, srv2067351549=48, srv1718988663=30, srv96827496=99, srv501170801=69, srv238131281=56, srv1386186356=14, srv838472890=88, srv1335512559=11, srv718642317=81, srv1964811004=43, srv2056978732=47, srv362276155=61, srv1462420104=18, srv93962040=95, srv1519379194=21, srv1386828976=15, srv613805597=75, srv1064502566=4, srv950302998=97, srv629436261=76, srv132656897=10, srv1502647285=20, srv893877218=91, srv757571264=82, srv1471525293=19, srv943491429=96, srv886849065=90, srv612667166=74, srv586640970=73, srv960839325=98, srv1816100852=34, srv1655817117=26, srv554178494=71, srv1900292005=42, srv1692635455=29, srv2096901958=50, srv1542987652=23, srv224016293=54, srv669897945=79, srv234653344=55, srv1675631891=28, srv1966212184=44, srv536893167=70, srv841153269=89, srv803811853=86, srv759649079=83, srv1536744005=22, srv898100257=92, srv1298972806=9, srv1343340583=12, srv650281721=77, srv797137943=85, srv1810731201=33, srv361611008=60, srv1744581560=31, srv1820968868=35, srv1881060607=39, srv1989760201=45, srv1272933730=8, srv366296420=62, srv1039392332=2, srv787825381=84, srv1062402423=3, srv350020447=58, srv264607481=57, srv109328012=6, srv1617535518=25, srv2074441794=49, srv660402926=78, srv2141753053=53, srv425987317=66, srv1899033489=41, srv1418065596=16, srv1087900447=5, srv2042349116=46, srv442335602=67, srv911716242=93, srv1663439608=27, srv1885054245=40, srv1455408605=17, srv442736921=68, srv388318647=64, srv367191275=63, srv925116579=94, srv694444285=80, srv555559515=72, srv1833627106=36, srv2134234089=52, srv1010394490=0, srv1162651654=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:56:08,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:56:08,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:56:08,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:56:08,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:56:08,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:56:08,457 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:56:08,458 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:56:08,459 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 08:56:08,460 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 08:56:08,461 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 08:56:08,462 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 08:56:08,463 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 08:56:08,464 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 08:56:08,465 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 08:56:08,466 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 08:56:08,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 08:56:08,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 08:56:08,467 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 08:56:08,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021167448015012646); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03164406423560035, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021167448015012646); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:08,589 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 08:56:08,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015331526329835745, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021167448015012646); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03164406423560035, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021167448015012646); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012083333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 08:56:08,817 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 332 ms to try 5000 different iterations.  Found a solution that moves 1002 regions; Going from a computed imbalance of 0.015331526329835745 to a new imbalance of 7.104844986760218E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060032829693241); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004854793875029461); MoveCostFunction : (multiplier=7.0, imbalance=0.11133333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060032829693241); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:08,819 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1298972806:82 , srv1966212184:82 , srv536893167:82 , srv629436261:82 , srv821919128:82 , srv1502647285:83 , srv1655817117:83 , srv1810731201:83 , srv1833627106:83 , srv1841886051:83 , srv1881060607:83 , srv1885054245:83 , srv366296420:83 , srv442335602:83 , srv943491429:83 , srv1335512559:84 , srv1343340583:84 , srv1359418044:84 , srv1462420104:84 , srv1617535518:84 , srv1663439608:84 , srv1675631891:84 , srv1744581560:84 , srv1900292005:84 , srv234653344:84 , srv264607481:84 , srv361611008:84 , srv555559515:84 , srv1010394490:85 , srv1064502566:85 , srv1087900447:85 , srv1272933730:85 , srv1455408605:85 , srv1536744005:85 , srv1542987652:85 , srv1816100852:85 , srv2056978732:85 , srv2131692509:85 , srv350020447:85 , srv362276155:85 , srv367191275:85 , srv388318647:85 , srv613805597:85 , srv797137943:85 , srv893877218:85 , srv911716242:85 , srv96827496:85 , srv1027512460:86 , srv1062402423:86 , srv109328012:86 , srv1386186356:86 , srv1418065596:86 , srv1471525293:86 , srv1519379194:86 , srv1802463753:86 , srv1820968868:86 , srv1964811004:86 , srv1989760201:86 , srv2067351549:86 , srv2074441794:86 , srv2096901958:86 , srv501170801:86 , srv554178494:86 , srv612667166:86 , srv650281721:86 , srv660402926:86 , srv669897945:86 , srv694444285:86 , srv757571264:86 , srv759649079:86 , srv787825381:86 , srv838472890:86 , srv841153269:86 , srv898100257:86 , srv93962040:86 , srv1039392332:87 , srv1591954769:87 , srv1692635455:87 , srv1718988663:87 , srv1859431326:87 , srv2042349116:87 , srv2141753053:87 , srv238131281:87 , srv352694159:87 , srv394960625:87 , srv425987317:87 , srv442736921:87 , srv718642317:87 , srv803811853:87 , srv925116579:87 , srv950302998:87 , srv1162651654:88 , srv132656897:88 , srv1386828976:88 , srv1899033489:88 , srv2134234089:88 , srv586640970:88 , srv886849065:88 , srv960839325:88 , srv224016293:549 }
2023-03-23 08:56:08,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:56:08,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1802463753=32, srv1027512460=1, srv1591954769=24, srv2131692509=51, srv821919128=87, srv1841886051=37, srv352694159=59, srv394960625=65, srv1859431326=38, srv1359418044=13, srv2067351549=48, srv1718988663=30, srv96827496=99, srv501170801=69, srv238131281=56, srv1386186356=14, srv838472890=88, srv1335512559=11, srv718642317=81, srv1964811004=43, srv2056978732=47, srv362276155=61, srv1462420104=18, srv93962040=95, srv1519379194=21, srv1386828976=15, srv613805597=75, srv1064502566=4, srv950302998=97, srv629436261=76, srv132656897=10, srv1502647285=20, srv893877218=91, srv757571264=82, srv1471525293=19, srv943491429=96, srv886849065=90, srv612667166=74, srv586640970=73, srv960839325=98, srv1816100852=34, srv1655817117=26, srv554178494=71, srv1900292005=42, srv1692635455=29, srv2096901958=50, srv1542987652=23, srv224016293=54, srv669897945=79, srv234653344=55, srv1675631891=28, srv1966212184=44, srv536893167=70, srv841153269=89, srv803811853=86, srv759649079=83, srv1536744005=22, srv898100257=92, srv1298972806=9, srv1343340583=12, srv650281721=77, srv797137943=85, srv1810731201=33, srv361611008=60, srv1744581560=31, srv1820968868=35, srv1881060607=39, srv1989760201=45, srv1272933730=8, srv366296420=62, srv1039392332=2, srv787825381=84, srv1062402423=3, srv350020447=58, srv264607481=57, srv109328012=6, srv1617535518=25, srv2074441794=49, srv660402926=78, srv2141753053=53, srv425987317=66, srv1899033489=41, srv1418065596=16, srv1087900447=5, srv2042349116=46, srv442335602=67, srv911716242=93, srv1663439608=27, srv1885054245=40, srv1455408605=17, srv442736921=68, srv388318647=64, srv367191275=63, srv925116579=94, srv694444285=80, srv555559515=72, srv1833627106=36, srv2134234089=52, srv1010394490=0, srv1162651654=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:56:08,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:56:08,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:56:08,833 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:56:08,834 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 08:56:08,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 08:56:08,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 08:56:08,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 08:56:08,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 08:56:08,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 08:56:08,942 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060032829693241); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004854793875029461); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060032829693241); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:08,942 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 08:56:08,942 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.034683447167562E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0060032829693241); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004854793875029461); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0060032829693241); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 08:56:09,273 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 423 ms to try 5000 different iterations.  Found a solution that moves 215 regions; Going from a computed imbalance of 7.034683447167562E-4 to a new imbalance of 1.1628429084902737E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.02388888888888889); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:09,274 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv841153269:82 , srv1418065596:83 , srv1744581560:83 , srv1810731201:83 , srv1859431326:83 , srv352694159:83 , srv886849065:83 , srv1027512460:84 , srv1087900447:84 , srv1162651654:84 , srv1359418044:84 , srv1386186356:84 , srv1462420104:84 , srv1502647285:84 , srv1536744005:84 , srv1591954769:84 , srv1663439608:84 , srv1692635455:84 , srv1718988663:84 , srv1802463753:84 , srv1899033489:84 , srv1964811004:84 , srv2042349116:84 , srv2074441794:84 , srv2096901958:84 , srv2134234089:84 , srv2141753053:84 , srv367191275:84 , srv442335602:84 , srv536893167:84 , srv554178494:84 , srv555559515:84 , srv586640970:84 , srv669897945:84 , srv718642317:84 , srv757571264:84 , srv759649079:84 , srv787825381:84 , srv797137943:84 , srv803811853:84 , srv898100257:84 , srv911716242:84 , srv93962040:84 , srv960839325:84 , srv1010394490:85 , srv1039392332:85 , srv1062402423:85 , srv1064502566:85 , srv109328012:85 , srv1272933730:85 , srv1298972806:85 , srv132656897:85 , srv1335512559:85 , srv1386828976:85 , srv1455408605:85 , srv1471525293:85 , srv1519379194:85 , srv1542987652:85 , srv1617535518:85 , srv1675631891:85 , srv1816100852:85 , srv1820968868:85 , srv1833627106:85 , srv1841886051:85 , srv1881060607:85 , srv1900292005:85 , srv1966212184:85 , srv1989760201:85 , srv2056978732:85 , srv2067351549:85 , srv2131692509:85 , srv234653344:85 , srv238131281:85 , srv264607481:85 , srv350020447:85 , srv361611008:85 , srv366296420:85 , srv388318647:85 , srv394960625:85 , srv425987317:85 , srv442736921:85 , srv501170801:85 , srv612667166:85 , srv613805597:85 , srv650281721:85 , srv660402926:85 , srv694444285:85 , srv821919128:85 , srv838472890:85 , srv893877218:85 , srv925116579:85 , srv943491429:85 , srv950302998:85 , srv96827496:85 , srv1343340583:86 , srv1655817117:86 , srv1885054245:86 , srv362276155:86 , srv629436261:86 , srv224016293:632 }
2023-03-23 08:56:09,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:56:09,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1802463753=32, srv1027512460=1, srv1591954769=24, srv2131692509=51, srv821919128=87, srv1841886051=37, srv352694159=59, srv394960625=65, srv1859431326=38, srv1359418044=13, srv2067351549=48, srv1718988663=30, srv96827496=99, srv501170801=69, srv238131281=56, srv1386186356=14, srv838472890=88, srv1335512559=11, srv718642317=81, srv1964811004=43, srv2056978732=47, srv362276155=61, srv1462420104=18, srv93962040=95, srv1519379194=21, srv1386828976=15, srv613805597=75, srv1064502566=4, srv950302998=97, srv629436261=76, srv132656897=10, srv1502647285=20, srv893877218=91, srv757571264=82, srv1471525293=19, srv943491429=96, srv886849065=90, srv612667166=74, srv586640970=73, srv960839325=98, srv1816100852=34, srv1655817117=26, srv554178494=71, srv1900292005=42, srv1692635455=29, srv2096901958=50, srv1542987652=23, srv224016293=54, srv669897945=79, srv234653344=55, srv1675631891=28, srv1966212184=44, srv536893167=70, srv841153269=89, srv803811853=86, srv759649079=83, srv1536744005=22, srv898100257=92, srv1298972806=9, srv1343340583=12, srv650281721=77, srv797137943=85, srv1810731201=33, srv361611008=60, srv1744581560=31, srv1820968868=35, srv1881060607=39, srv1989760201=45, srv1272933730=8, srv366296420=62, srv1039392332=2, srv787825381=84, srv1062402423=3, srv350020447=58, srv264607481=57, srv109328012=6, srv1617535518=25, srv2074441794=49, srv660402926=78, srv2141753053=53, srv425987317=66, srv1899033489=41, srv1418065596=16, srv1087900447=5, srv2042349116=46, srv442335602=67, srv911716242=93, srv1663439608=27, srv1885054245=40, srv1455408605=17, srv442736921=68, srv388318647=64, srv367191275=63, srv925116579=94, srv694444285=80, srv555559515=72, srv1833627106=36, srv2134234089=52, srv1010394490=0, srv1162651654=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:56:09,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:56:09,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:56:09,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:56:09,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:56:09,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 08:56:09,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 08:56:09,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 08:56:09,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 08:56:09,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 08:56:09,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 08:56:09,399 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.0122966908014432E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:09,400 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv841153269:82 , srv1418065596:83 , srv1744581560:83 , srv1810731201:83 , srv1859431326:83 , srv352694159:83 , srv886849065:83 , srv1027512460:84 , srv1087900447:84 , srv1162651654:84 , srv1359418044:84 , srv1386186356:84 , srv1462420104:84 , srv1502647285:84 , srv1536744005:84 , srv1591954769:84 , srv1663439608:84 , srv1692635455:84 , srv1718988663:84 , srv1802463753:84 , srv1899033489:84 , srv1964811004:84 , srv2042349116:84 , srv2074441794:84 , srv2096901958:84 , srv2134234089:84 , srv2141753053:84 , srv367191275:84 , srv442335602:84 , srv536893167:84 , srv554178494:84 , srv555559515:84 , srv586640970:84 , srv669897945:84 , srv718642317:84 , srv757571264:84 , srv759649079:84 , srv787825381:84 , srv797137943:84 , srv803811853:84 , srv898100257:84 , srv911716242:84 , srv93962040:84 , srv960839325:84 , srv1010394490:85 , srv1039392332:85 , srv1062402423:85 , srv1064502566:85 , srv109328012:85 , srv1272933730:85 , srv1298972806:85 , srv132656897:85 , srv1335512559:85 , srv1386828976:85 , srv1455408605:85 , srv1471525293:85 , srv1519379194:85 , srv1542987652:85 , srv1617535518:85 , srv1675631891:85 , srv1816100852:85 , srv1820968868:85 , srv1833627106:85 , srv1841886051:85 , srv1881060607:85 , srv1900292005:85 , srv1966212184:85 , srv1989760201:85 , srv2056978732:85 , srv2067351549:85 , srv2131692509:85 , srv234653344:85 , srv238131281:85 , srv264607481:85 , srv350020447:85 , srv361611008:85 , srv366296420:85 , srv388318647:85 , srv394960625:85 , srv425987317:85 , srv442736921:85 , srv501170801:85 , srv612667166:85 , srv613805597:85 , srv650281721:85 , srv660402926:85 , srv694444285:85 , srv821919128:85 , srv838472890:85 , srv893877218:85 , srv925116579:85 , srv943491429:85 , srv950302998:85 , srv96827496:85 , srv1343340583:86 , srv1655817117:86 , srv1885054245:86 , srv362276155:86 , srv629436261:86 , srv224016293:632 }
2023-03-23 08:56:09,423 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=350 (was 303) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=6083 (was 6559)
2023-03-23 08:56:09,432 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=350, ProcessCount=169, AvailableMemoryMB=6082
2023-03-23 08:56:09,433 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 08:56:09,433 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-23 08:56:09,443 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011183625:84 , srv1014152246:84 , srv1040617994:84 , srv1151081852:84 , srv1262175335:84 , srv1402487883:84 , srv1469939274:84 , srv1549170823:84 , srv1556901770:84 , srv1597610745:84 , srv1704416636:84 , srv1726516602:84 , srv1773915744:84 , srv183049174:84 , srv310831876:84 , srv331454210:84 , srv336722733:84 , srv414507358:84 , srv417960566:84 , srv423094803:84 , srv472380397:84 , srv519758674:84 , srv593160145:84 , srv653056878:84 , srv70909154:84 , srv727225905:84 , srv803779910:84 , srv96034987:84 , srv970211233:84 , srv1444005164:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-23 08:56:09,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:56:09,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1773915744=13, srv593160145=23, srv1444005164=6, srv803779910=27, srv423094803=20, srv1402487883=5, srv472380397=21, srv1726516602=12, srv70909154=25, srv1040617994=2, srv331454210=16, srv1549170823=8, srv1556901770=9, srv1597610745=10, srv970211233=29, srv183049174=14, srv519758674=22, srv1014152246=1, srv727225905=26, srv96034987=28, srv310831876=15, srv414507358=18, srv1704416636=11, srv1469939274=7, srv417960566=19, srv1151081852=3, srv653056878=24, srv1262175335=4, srv1011183625=0, srv336722733=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:56:09,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:56:09,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:56:09,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:56:09,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:56:09,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:56:09,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:56:09,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:56:09,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-23 08:56:09,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2037527084832842, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:56:09,470 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912835061395571, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2037527084832842, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-23 08:58:09,459 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5793520 different iterations.  Found a solution that moves 1861 regions; Going from a computed imbalance of 0.9912835061395571 to a new imbalance of 4.3436668390529226E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6892592592592592, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:58:09,486 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1011183625:90 , srv1014152246:90 , srv1040617994:90 , srv1151081852:90 , srv1262175335:90 , srv1402487883:90 , srv1444005164:90 , srv1469939274:90 , srv1549170823:90 , srv1556901770:90 , srv1597610745:90 , srv1704416636:90 , srv1726516602:90 , srv1773915744:90 , srv183049174:90 , srv310831876:90 , srv331454210:90 , srv336722733:90 , srv414507358:90 , srv417960566:90 , srv423094803:90 , srv472380397:90 , srv519758674:90 , srv593160145:90 , srv653056878:90 , srv70909154:90 , srv727225905:90 , srv803779910:90 , srv96034987:90 , srv970211233:90 }
2023-03-23 08:58:09,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 08:58:09,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1773915744=13, srv593160145=23, srv1444005164=6, srv803779910=27, srv423094803=20, srv1402487883=5, srv472380397=21, srv1726516602=12, srv70909154=25, srv1040617994=2, srv331454210=16, srv1549170823=8, srv1556901770=9, srv1597610745=10, srv970211233=29, srv183049174=14, srv519758674=22, srv1014152246=1, srv727225905=26, srv96034987=28, srv310831876=15, srv414507358=18, srv1704416636=11, srv1469939274=7, srv417960566=19, srv1151081852=3, srv653056878=24, srv1262175335=4, srv1011183625=0, srv336722733=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 08:58:09,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 08:58:09,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 08:58:09,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-23 08:58:09,521 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 08:58:09,522 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1011183625:90 , srv1014152246:90 , srv1040617994:90 , srv1151081852:90 , srv1262175335:90 , srv1402487883:90 , srv1444005164:90 , srv1469939274:90 , srv1549170823:90 , srv1556901770:90 , srv1597610745:90 , srv1704416636:90 , srv1726516602:90 , srv1773915744:90 , srv183049174:90 , srv310831876:90 , srv331454210:90 , srv336722733:90 , srv414507358:90 , srv417960566:90 , srv423094803:90 , srv472380397:90 , srv519758674:90 , srv593160145:90 , srv653056878:90 , srv70909154:90 , srv727225905:90 , srv803779910:90 , srv96034987:90 , srv970211233:90 }
2023-03-23 08:58:09,537 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=366 (was 350) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 169) - ProcessCount LEAK? -, AvailableMemoryMB=5695 (was 6082)