2023-05-05 15:53:35,753 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-05 15:53:36,083 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-05 15:53:36,196 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-05 15:53:36,264 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-05 15:53:36,272 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-05 15:53:36,285 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=179, AvailableMemoryMB=6637
2023-05-05 15:53:36,287 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-05 15:53:36,287 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-05 15:53:36,420 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002813108:84 , srv1031589611:84 , srv1046122213:84 , srv1093078777:84 , srv1093739828:84 , srv1098395931:84 , srv1117093423:84 , srv11323319:84 , srv1149260961:84 , srv1158823477:84 , srv1170056407:84 , srv1178455450:84 , srv118979206:84 , srv1251238167:84 , srv1260333602:84 , srv1312360475:84 , srv1315256775:84 , srv1332042352:84 , srv1439329535:84 , srv1443247829:84 , srv1455398647:84 , srv1456570808:84 , srv1485873399:84 , srv1490271233:84 , srv1503321609:84 , srv1516085936:84 , srv1573559226:84 , srv1580391217:84 , srv1580549134:84 , srv1580563185:84 , srv1603428409:84 , srv1609662628:84 , srv1623681351:84 , srv1631266655:84 , srv1634636808:84 , srv1696716232:84 , srv1706147715:84 , srv1713440546:84 , srv1728007067:84 , srv1762512096:84 , srv1848738972:84 , srv1869700516:84 , srv1875247146:84 , srv1875748893:84 , srv1899957551:84 , srv1909672333:84 , srv1910763743:84 , srv1924892417:84 , srv1936371345:84 , srv19483478:84 , srv1974630146:84 , srv1985094867:84 , srv1993633753:84 , srv1999813323:84 , srv2022072653:84 , srv2043087176:84 , srv2102953440:84 , srv2104696355:84 , srv2120488107:84 , srv2140156471:84 , srv239641880:84 , srv241399702:84 , srv255597364:84 , srv291700199:84 , srv30379575:84 , srv309343592:84 , srv320740777:84 , srv323776586:84 , srv32570759:84 , srv327936480:84 , srv374153856:84 , srv374755388:84 , srv384864453:84 , srv443972030:84 , srv466039233:84 , srv470589531:84 , srv473360763:84 , srv488144053:84 , srv48884403:84 , srv545681940:84 , srv602273411:84 , srv631432974:84 , srv636431025:84 , srv673608427:84 , srv710286501:84 , srv711316236:84 , srv723398121:84 , srv759865138:84 , srv770103462:84 , srv780449553:84 , srv783535854:84 , srv802653840:84 , srv854153801:84 , srv861595971:84 , srv873658994:84 , srv884980484:84 , srv893107185:84 , srv903433874:84 , srv989793695:84 , srv1789282461:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-05 15:53:36,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:36,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv780449553=90, srv1149260961=8, srv1634636808=34, srv1706147715=36, srv239641880=61, srv861595971=94, srv1713440546=37, srv374755388=72, srv1848738972=41, srv631432974=82, srv636431025=83, srv1974630146=51, srv1490271233=23, srv545681940=80, srv1909672333=46, srv602273411=81, srv255597364=63, srv48884403=79, srv241399702=62, srv1603428409=30, srv1936371345=49, srv1170056407=10, srv470589531=76, srv1312360475=15, srv1762512096=39, srv1789282461=40, srv320740777=67, srv488144053=78, srv802653840=92, srv327936480=70, srv783535854=91, srv884980484=96, srv989793695=99, srv1046122213=2, srv1158823477=9, srv473360763=77, srv1910763743=47, srv1728007067=38, srv893107185=97, srv1875247146=43, srv1485873399=22, srv2140156471=60, srv1875748893=44, srv759865138=88, srv11323319=7, srv1093739828=4, srv1503321609=24, srv1609662628=31, srv1993633753=53, srv1899957551=45, srv1455398647=20, srv1031589611=1, srv1580391217=27, srv118979206=12, srv711316236=86, srv323776586=68, srv903433874=98, srv1516085936=25, srv1098395931=5, srv1251238167=13, srv2022072653=55, srv1696716232=35, srv1580549134=28, srv2120488107=59, srv1573559226=26, srv770103462=89, srv723398121=87, srv1315256775=16, srv854153801=93, srv2043087176=56, srv1924892417=48, srv710286501=85, srv1439329535=18, srv1580563185=29, srv374153856=71, srv1093078777=3, srv291700199=64, srv1117093423=6, srv1623681351=32, srv1985094867=52, srv1332042352=17, srv30379575=65, srv309343592=66, srv466039233=75, srv19483478=50, srv1869700516=42, srv673608427=84, srv2104696355=58, srv1631266655=33, srv1260333602=14, srv384864453=73, srv1178455450=11, srv1999813323=54, srv443972030=74, srv1002813108=0, srv1443247829=19, srv873658994=95, srv1456570808=21, srv2102953440=57, srv32570759=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:36,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:36,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:36,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:36,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:36,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:36,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:36,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:36,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:36,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:36,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:36,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:36,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:36,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:36,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:36,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:36,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:36,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:36,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:36,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:36,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:36,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:36,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:36,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:36,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:36,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:36,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:36,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:36,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:36,552 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:36,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:36,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-05 15:53:36,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-05 15:53:36,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-05 15:53:36,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-05 15:53:36,553 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-05 15:53:36,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-05 15:53:36,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-05 15:53:36,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-05 15:53:36,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-05 15:53:36,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-05 15:53:36,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-05 15:53:36,555 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-05 15:53:36,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-05 15:53:36,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-05 15:53:36,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-05 15:53:36,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-05 15:53:36,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-05 15:53:36,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-05 15:53:36,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-05 15:53:36,558 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-05 15:53:36,559 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-05 15:53:36,560 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-05 15:53:36,561 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-05 15:53:36,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-05 15:53:36,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-05 15:53:36,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-05 15:53:36,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-05 15:53:36,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:36,881 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-05 15:53:36,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-05 15:53:37,348 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 754 ms to try 5000 different iterations.  Found a solution that moves 4683 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1812630281641901. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039127215845596634, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05082263783270059, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5203333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039127215845596634, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24341666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:37,372 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1046122213:82 , srv1098395931:82 , srv1762512096:82 , srv384864453:82 , srv443972030:82 , srv466039233:82 , srv723398121:82 , srv1332042352:83 , srv1485873399:83 , srv1696716232:83 , srv1713440546:83 , srv1875247146:83 , srv1899957551:83 , srv1910763743:83 , srv1936371345:83 , srv241399702:83 , srv32570759:83 , srv374755388:83 , srv488144053:83 , srv873658994:83 , srv884980484:83 , srv989793695:83 , srv1093739828:84 , srv11323319:84 , srv118979206:84 , srv1260333602:84 , srv1634636808:84 , srv1728007067:84 , srv327936480:84 , srv631432974:84 , srv710286501:84 , srv1002813108:85 , srv1093078777:85 , srv1443247829:85 , srv1490271233:85 , srv1516085936:85 , srv1993633753:85 , srv2022072653:85 , srv2104696355:85 , srv239641880:85 , srv255597364:85 , srv30379575:85 , srv320740777:85 , srv636431025:85 , srv780449553:85 , srv854153801:85 , srv1031589611:86 , srv1158823477:86 , srv1312360475:86 , srv1456570808:86 , srv1573559226:86 , srv1580563185:86 , srv1603428409:86 , srv1609662628:86 , srv1706147715:86 , srv1875748893:86 , srv2043087176:86 , srv770103462:86 , srv893107185:86 , srv1455398647:87 , srv1869700516:87 , srv1974630146:87 , srv1999813323:87 , srv291700199:87 , srv1117093423:88 , srv1178455450:88 , srv1924892417:88 , srv2120488107:88 , srv309343592:88 , srv323776586:88 , srv470589531:88 , srv1251238167:89 , srv1315256775:89 , srv1580391217:89 , srv1848738972:89 , srv1909672333:89 , srv2102953440:89 , srv374153856:89 , srv759865138:89 , srv783535854:89 , srv903433874:89 , srv1149260961:90 , srv1503321609:90 , srv1985094867:90 , srv2140156471:90 , srv711316236:90 , srv802653840:90 , srv1170056407:91 , srv1623681351:91 , srv1631266655:91 , srv473360763:91 , srv1580549134:92 , srv19483478:92 , srv48884403:92 , srv602273411:92 , srv545681940:93 , srv673608427:93 , srv1439329535:95 , srv861595971:101 , srv1789282461:437 }
2023-05-05 15:53:37,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:37,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv780449553=90, srv1149260961=8, srv1634636808=34, srv1706147715=36, srv239641880=61, srv861595971=94, srv1713440546=37, srv374755388=72, srv1848738972=41, srv631432974=82, srv636431025=83, srv1974630146=51, srv1490271233=23, srv545681940=80, srv1909672333=46, srv602273411=81, srv255597364=63, srv48884403=79, srv241399702=62, srv1603428409=30, srv1936371345=49, srv1170056407=10, srv470589531=76, srv1312360475=15, srv1762512096=39, srv1789282461=40, srv320740777=67, srv488144053=78, srv802653840=92, srv327936480=70, srv783535854=91, srv884980484=96, srv989793695=99, srv1046122213=2, srv1158823477=9, srv473360763=77, srv1910763743=47, srv1728007067=38, srv893107185=97, srv1875247146=43, srv1485873399=22, srv2140156471=60, srv1875748893=44, srv759865138=88, srv11323319=7, srv1093739828=4, srv1503321609=24, srv1609662628=31, srv1993633753=53, srv1899957551=45, srv1455398647=20, srv1031589611=1, srv1580391217=27, srv118979206=12, srv711316236=86, srv323776586=68, srv903433874=98, srv1516085936=25, srv1098395931=5, srv1251238167=13, srv2022072653=55, srv1696716232=35, srv1580549134=28, srv2120488107=59, srv1573559226=26, srv770103462=89, srv723398121=87, srv1315256775=16, srv854153801=93, srv2043087176=56, srv1924892417=48, srv710286501=85, srv1439329535=18, srv1580563185=29, srv374153856=71, srv1093078777=3, srv291700199=64, srv1117093423=6, srv1623681351=32, srv1985094867=52, srv1332042352=17, srv30379575=65, srv309343592=66, srv466039233=75, srv19483478=50, srv1869700516=42, srv673608427=84, srv2104696355=58, srv1631266655=33, srv1260333602=14, srv384864453=73, srv1178455450=11, srv1999813323=54, srv443972030=74, srv1002813108=0, srv1443247829=19, srv873658994=95, srv1456570808=21, srv2102953440=57, srv32570759=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:37,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:37,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:37,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:37,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:37,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:37,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-05 15:53:37,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-05 15:53:37,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-05 15:53:37,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-05 15:53:37,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-05 15:53:37,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-05 15:53:37,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-05 15:53:37,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-05 15:53:37,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-05 15:53:37,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-05 15:53:37,557 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039127215845596634, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05082263783270059, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039127215845596634, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24341666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:37,557 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-05 15:53:37,558 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18123023709733257, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039127215845596634, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05082263783270059, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039127215845596634, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1765, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24341666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-05 15:53:37,915 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 482 ms to try 5000 different iterations.  Found a solution that moves 2665 regions; Going from a computed imbalance of 0.18123023709733257 to a new imbalance of 0.015185141653943924. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019812690656779295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02976149109861633, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2961111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019812690656779295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:37,925 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv861595971:70 , srv1439329535:79 , srv19483478:79 , srv473360763:79 , srv1503321609:80 , srv1580549134:80 , srv48884403:80 , srv602273411:80 , srv673608427:80 , srv1631266655:81 , srv903433874:81 , srv1251238167:82 , srv1315256775:82 , srv1580391217:82 , srv1623681351:82 , srv1985094867:82 , srv2140156471:82 , srv309343592:83 , srv545681940:83 , srv759865138:83 , srv802653840:83 , srv1149260961:84 , srv1170056407:84 , srv1909672333:84 , srv1924892417:84 , srv2102953440:84 , srv2120488107:84 , srv291700199:84 , srv374153856:84 , srv470589531:84 , srv783535854:84 , srv1158823477:85 , srv1178455450:85 , srv1456570808:85 , srv1848738972:85 , srv1869700516:85 , srv323776586:85 , srv711316236:85 , srv770103462:85 , srv854153801:85 , srv1002813108:86 , srv1117093423:86 , srv118979206:86 , srv1312360475:86 , srv1455398647:86 , srv1609662628:86 , srv1706147715:86 , srv1974630146:86 , srv1999813323:86 , srv255597364:86 , srv893107185:86 , srv1031589611:87 , srv11323319:87 , srv1485873399:87 , srv1516085936:87 , srv1573559226:87 , srv1580563185:87 , srv1603428409:87 , srv1634636808:87 , srv1696716232:87 , srv1875748893:87 , srv2043087176:87 , srv2104696355:87 , srv239641880:87 , srv327936480:87 , srv631432974:87 , srv636431025:87 , srv780449553:87 , srv1046122213:88 , srv1260333602:88 , srv1443247829:88 , srv1713440546:88 , srv1728007067:88 , srv1762512096:88 , srv1899957551:88 , srv1910763743:88 , srv1993633753:88 , srv2022072653:88 , srv32570759:88 , srv443972030:88 , srv723398121:88 , srv1093078777:89 , srv1332042352:89 , srv1490271233:89 , srv241399702:89 , srv30379575:89 , srv320740777:89 , srv384864453:89 , srv873658994:89 , srv1875247146:90 , srv1936371345:90 , srv374755388:90 , srv488144053:90 , srv710286501:90 , srv1098395931:91 , srv989793695:91 , srv1093739828:92 , srv466039233:92 , srv884980484:92 , srv1789282461:513 }
2023-05-05 15:53:37,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:37,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv780449553=90, srv1149260961=8, srv1634636808=34, srv1706147715=36, srv239641880=61, srv861595971=94, srv1713440546=37, srv374755388=72, srv1848738972=41, srv631432974=82, srv636431025=83, srv1974630146=51, srv1490271233=23, srv545681940=80, srv1909672333=46, srv602273411=81, srv255597364=63, srv48884403=79, srv241399702=62, srv1603428409=30, srv1936371345=49, srv1170056407=10, srv470589531=76, srv1312360475=15, srv1762512096=39, srv1789282461=40, srv320740777=67, srv488144053=78, srv802653840=92, srv327936480=70, srv783535854=91, srv884980484=96, srv989793695=99, srv1046122213=2, srv1158823477=9, srv473360763=77, srv1910763743=47, srv1728007067=38, srv893107185=97, srv1875247146=43, srv1485873399=22, srv2140156471=60, srv1875748893=44, srv759865138=88, srv11323319=7, srv1093739828=4, srv1503321609=24, srv1609662628=31, srv1993633753=53, srv1899957551=45, srv1455398647=20, srv1031589611=1, srv1580391217=27, srv118979206=12, srv711316236=86, srv323776586=68, srv903433874=98, srv1516085936=25, srv1098395931=5, srv1251238167=13, srv2022072653=55, srv1696716232=35, srv1580549134=28, srv2120488107=59, srv1573559226=26, srv770103462=89, srv723398121=87, srv1315256775=16, srv854153801=93, srv2043087176=56, srv1924892417=48, srv710286501=85, srv1439329535=18, srv1580563185=29, srv374153856=71, srv1093078777=3, srv291700199=64, srv1117093423=6, srv1623681351=32, srv1985094867=52, srv1332042352=17, srv30379575=65, srv309343592=66, srv466039233=75, srv19483478=50, srv1869700516=42, srv673608427=84, srv2104696355=58, srv1631266655=33, srv1260333602=14, srv384864453=73, srv1178455450=11, srv1999813323=54, srv443972030=74, srv1002813108=0, srv1443247829=19, srv873658994=95, srv1456570808=21, srv2102953440=57, srv32570759=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:37,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:37,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-05 15:53:37,950 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-05 15:53:37,951 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-05 15:53:37,952 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-05 15:53:37,953 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-05 15:53:37,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-05 15:53:37,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-05 15:53:38,077 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019812690656779295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02976149109861633, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019812690656779295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:38,077 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-05 15:53:38,078 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015166480925100171, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019812690656779295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02976149109861633, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019812690656779295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04175, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-05 15:53:38,411 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 442 ms to try 5000 different iterations.  Found a solution that moves 1005 regions; Going from a computed imbalance of 0.015166480925100171 to a new imbalance of 7.26617819377574E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00432210984305678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005233064180601462); MoveCostFunction : (multiplier=7.0, imbalance=0.11166666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00432210984305678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:38,414 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1093739828:82 , srv545681940:82 , srv711316236:82 , srv1093078777:83 , srv1117093423:83 , srv1149260961:83 , srv1170056407:83 , srv320740777:83 , srv323776586:83 , srv710286501:83 , srv884980484:83 , srv1031589611:84 , srv1178455450:84 , srv1439329535:84 , srv1443247829:84 , srv1455398647:84 , srv1573559226:84 , srv1623681351:84 , srv1875247146:84 , srv1875748893:84 , srv1909672333:84 , srv1999813323:84 , srv2043087176:84 , srv30379575:84 , srv374755388:84 , srv466039233:84 , srv488144053:84 , srv673608427:84 , srv802653840:84 , srv989793695:84 , srv1098395931:85 , srv1312360475:85 , srv1490271233:85 , srv1580549134:85 , srv1603428409:85 , srv1706147715:85 , srv1848738972:85 , srv1936371345:85 , srv1974630146:85 , srv1985094867:85 , srv2022072653:85 , srv2102953440:85 , srv2104696355:85 , srv2120488107:85 , srv241399702:85 , srv255597364:85 , srv374153856:85 , srv780449553:85 , srv783535854:85 , srv1002813108:86 , srv1158823477:86 , srv1260333602:86 , srv1315256775:86 , srv1332042352:86 , srv1516085936:86 , srv1580563185:86 , srv1631266655:86 , srv1713440546:86 , srv1728007067:86 , srv1910763743:86 , srv1924892417:86 , srv19483478:86 , srv1993633753:86 , srv2140156471:86 , srv239641880:86 , srv470589531:86 , srv48884403:86 , srv602273411:86 , srv636431025:86 , srv759865138:86 , srv861595971:86 , srv873658994:86 , srv893107185:86 , srv11323319:87 , srv118979206:87 , srv1251238167:87 , srv1456570808:87 , srv1485873399:87 , srv1580391217:87 , srv1609662628:87 , srv1634636808:87 , srv1869700516:87 , srv1899957551:87 , srv291700199:87 , srv309343592:87 , srv32570759:87 , srv327936480:87 , srv384864453:87 , srv443972030:87 , srv631432974:87 , srv723398121:87 , srv770103462:87 , srv1046122213:88 , srv1503321609:88 , srv1696716232:88 , srv1762512096:88 , srv473360763:88 , srv854153801:88 , srv903433874:88 , srv1789282461:546 }
2023-05-05 15:53:38,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:38,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv780449553=90, srv1149260961=8, srv1634636808=34, srv1706147715=36, srv239641880=61, srv861595971=94, srv1713440546=37, srv374755388=72, srv1848738972=41, srv631432974=82, srv636431025=83, srv1974630146=51, srv1490271233=23, srv545681940=80, srv1909672333=46, srv602273411=81, srv255597364=63, srv48884403=79, srv241399702=62, srv1603428409=30, srv1936371345=49, srv1170056407=10, srv470589531=76, srv1312360475=15, srv1762512096=39, srv1789282461=40, srv320740777=67, srv488144053=78, srv802653840=92, srv327936480=70, srv783535854=91, srv884980484=96, srv989793695=99, srv1046122213=2, srv1158823477=9, srv473360763=77, srv1910763743=47, srv1728007067=38, srv893107185=97, srv1875247146=43, srv1485873399=22, srv2140156471=60, srv1875748893=44, srv759865138=88, srv11323319=7, srv1093739828=4, srv1503321609=24, srv1609662628=31, srv1993633753=53, srv1899957551=45, srv1455398647=20, srv1031589611=1, srv1580391217=27, srv118979206=12, srv711316236=86, srv323776586=68, srv903433874=98, srv1516085936=25, srv1098395931=5, srv1251238167=13, srv2022072653=55, srv1696716232=35, srv1580549134=28, srv2120488107=59, srv1573559226=26, srv770103462=89, srv723398121=87, srv1315256775=16, srv854153801=93, srv2043087176=56, srv1924892417=48, srv710286501=85, srv1439329535=18, srv1580563185=29, srv374153856=71, srv1093078777=3, srv291700199=64, srv1117093423=6, srv1623681351=32, srv1985094867=52, srv1332042352=17, srv30379575=65, srv309343592=66, srv466039233=75, srv19483478=50, srv1869700516=42, srv673608427=84, srv2104696355=58, srv1631266655=33, srv1260333602=14, srv384864453=73, srv1178455450=11, srv1999813323=54, srv443972030=74, srv1002813108=0, srv1443247829=19, srv873658994=95, srv1456570808=21, srv2102953440=57, srv32570759=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:38,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:38,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-05 15:53:38,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-05 15:53:38,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-05 15:53:38,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-05 15:53:38,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-05 15:53:38,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-05 15:53:38,442 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-05 15:53:38,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-05 15:53:38,444 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-05 15:53:38,581 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00432210984305678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005233064180601462); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00432210984305678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:38,582 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-05 15:53:38,582 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.195806589693286E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00432210984305678); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005233064180601462); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.00432210984305678); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0033333333333333335); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-05 15:53:38,939 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 479 ms to try 5000 different iterations.  Found a solution that moves 209 regions; Going from a computed imbalance of 7.195806589693286E-4 to a new imbalance of 2.7323093035496678E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.02322222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:38,940 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1490271233:83 , srv1580563185:83 , srv1609662628:83 , srv1869700516:83 , srv1993633753:83 , srv291700199:83 , srv1046122213:84 , srv1093739828:84 , srv11323319:84 , srv1170056407:84 , srv1251238167:84 , srv1260333602:84 , srv1332042352:84 , srv1439329535:84 , srv1456570808:84 , srv1503321609:84 , srv1516085936:84 , srv1580391217:84 , srv1603428409:84 , srv1631266655:84 , srv1634636808:84 , srv1696716232:84 , srv1728007067:84 , srv1762512096:84 , srv1848738972:84 , srv1875247146:84 , srv1899957551:84 , srv1924892417:84 , srv1936371345:84 , srv1974630146:84 , srv2022072653:84 , srv2102953440:84 , srv2140156471:84 , srv239641880:84 , srv30379575:84 , srv309343592:84 , srv32570759:84 , srv327936480:84 , srv374153856:84 , srv466039233:84 , srv470589531:84 , srv473360763:84 , srv48884403:84 , srv545681940:84 , srv602273411:84 , srv631432974:84 , srv636431025:84 , srv673608427:84 , srv759865138:84 , srv770103462:84 , srv783535854:84 , srv854153801:84 , srv873658994:84 , srv884980484:84 , srv893107185:84 , srv903433874:84 , srv989793695:84 , srv1002813108:85 , srv1031589611:85 , srv1093078777:85 , srv1098395931:85 , srv1117093423:85 , srv1149260961:85 , srv1158823477:85 , srv1178455450:85 , srv118979206:85 , srv1312360475:85 , srv1315256775:85 , srv1443247829:85 , srv1455398647:85 , srv1485873399:85 , srv1573559226:85 , srv1580549134:85 , srv1623681351:85 , srv1706147715:85 , srv1713440546:85 , srv1875748893:85 , srv1909672333:85 , srv1910763743:85 , srv19483478:85 , srv1985094867:85 , srv1999813323:85 , srv2043087176:85 , srv2104696355:85 , srv2120488107:85 , srv241399702:85 , srv320740777:85 , srv374755388:85 , srv384864453:85 , srv443972030:85 , srv488144053:85 , srv710286501:85 , srv711316236:85 , srv723398121:85 , srv780449553:85 , srv802653840:85 , srv861595971:85 , srv255597364:86 , srv323776586:86 , srv1789282461:646 }
2023-05-05 15:53:38,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:38,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv780449553=90, srv1149260961=8, srv1634636808=34, srv1706147715=36, srv239641880=61, srv861595971=94, srv1713440546=37, srv374755388=72, srv1848738972=41, srv631432974=82, srv636431025=83, srv1974630146=51, srv1490271233=23, srv545681940=80, srv1909672333=46, srv602273411=81, srv255597364=63, srv48884403=79, srv241399702=62, srv1603428409=30, srv1936371345=49, srv1170056407=10, srv470589531=76, srv1312360475=15, srv1762512096=39, srv1789282461=40, srv320740777=67, srv488144053=78, srv802653840=92, srv327936480=70, srv783535854=91, srv884980484=96, srv989793695=99, srv1046122213=2, srv1158823477=9, srv473360763=77, srv1910763743=47, srv1728007067=38, srv893107185=97, srv1875247146=43, srv1485873399=22, srv2140156471=60, srv1875748893=44, srv759865138=88, srv11323319=7, srv1093739828=4, srv1503321609=24, srv1609662628=31, srv1993633753=53, srv1899957551=45, srv1455398647=20, srv1031589611=1, srv1580391217=27, srv118979206=12, srv711316236=86, srv323776586=68, srv903433874=98, srv1516085936=25, srv1098395931=5, srv1251238167=13, srv2022072653=55, srv1696716232=35, srv1580549134=28, srv2120488107=59, srv1573559226=26, srv770103462=89, srv723398121=87, srv1315256775=16, srv854153801=93, srv2043087176=56, srv1924892417=48, srv710286501=85, srv1439329535=18, srv1580563185=29, srv374153856=71, srv1093078777=3, srv291700199=64, srv1117093423=6, srv1623681351=32, srv1985094867=52, srv1332042352=17, srv30379575=65, srv309343592=66, srv466039233=75, srv19483478=50, srv1869700516=42, srv673608427=84, srv2104696355=58, srv1631266655=33, srv1260333602=14, srv384864453=73, srv1178455450=11, srv1999813323=54, srv443972030=74, srv1002813108=0, srv1443247829=19, srv873658994=95, srv1456570808=21, srv2102953440=57, srv32570759=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:38,954 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:38,955 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-05 15:53:38,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-05 15:53:38,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-05 15:53:38,958 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-05 15:53:38,959 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-05 15:53:38,960 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-05 15:53:38,961 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-05 15:53:38,962 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-05 15:53:38,963 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-05 15:53:39,075 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:39,076 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-05 15:53:39,076 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.5859643756568046E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=2.735366466207394E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=2.735366466207394E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-05 15:53:39,446 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 475 ms to try 5000 different iterations.  Found a solution that moves 20 regions; Going from a computed imbalance of 2.5859643756568046E-5 to a new imbalance of 7.527800493511471E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0022222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:39,447 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1098395931:83 , srv1848738972:83 , srv384864453:83 , srv1002813108:84 , srv1031589611:84 , srv1046122213:84 , srv1093078777:84 , srv1093739828:84 , srv1117093423:84 , srv11323319:84 , srv1149260961:84 , srv1158823477:84 , srv1170056407:84 , srv1178455450:84 , srv118979206:84 , srv1251238167:84 , srv1260333602:84 , srv1312360475:84 , srv1315256775:84 , srv1332042352:84 , srv1439329535:84 , srv1443247829:84 , srv1455398647:84 , srv1456570808:84 , srv1485873399:84 , srv1490271233:84 , srv1503321609:84 , srv1516085936:84 , srv1573559226:84 , srv1580391217:84 , srv1580549134:84 , srv1580563185:84 , srv1603428409:84 , srv1609662628:84 , srv1623681351:84 , srv1631266655:84 , srv1634636808:84 , srv1696716232:84 , srv1706147715:84 , srv1713440546:84 , srv1728007067:84 , srv1762512096:84 , srv1869700516:84 , srv1875748893:84 , srv1899957551:84 , srv1909672333:84 , srv1910763743:84 , srv1924892417:84 , srv1936371345:84 , srv19483478:84 , srv1974630146:84 , srv1985094867:84 , srv1993633753:84 , srv1999813323:84 , srv2022072653:84 , srv2043087176:84 , srv2102953440:84 , srv2104696355:84 , srv2120488107:84 , srv2140156471:84 , srv239641880:84 , srv241399702:84 , srv255597364:84 , srv30379575:84 , srv309343592:84 , srv320740777:84 , srv323776586:84 , srv32570759:84 , srv327936480:84 , srv374153856:84 , srv374755388:84 , srv443972030:84 , srv466039233:84 , srv470589531:84 , srv473360763:84 , srv488144053:84 , srv48884403:84 , srv545681940:84 , srv602273411:84 , srv631432974:84 , srv636431025:84 , srv710286501:84 , srv711316236:84 , srv723398121:84 , srv759865138:84 , srv770103462:84 , srv780449553:84 , srv783535854:84 , srv802653840:84 , srv854153801:84 , srv861595971:84 , srv873658994:84 , srv884980484:84 , srv893107185:84 , srv903433874:84 , srv989793695:84 , srv1875247146:85 , srv291700199:85 , srv673608427:85 , srv1789282461:684 }
2023-05-05 15:53:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:39,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv780449553=90, srv1149260961=8, srv1634636808=34, srv1706147715=36, srv239641880=61, srv861595971=94, srv1713440546=37, srv374755388=72, srv1848738972=41, srv631432974=82, srv636431025=83, srv1974630146=51, srv1490271233=23, srv545681940=80, srv1909672333=46, srv602273411=81, srv255597364=63, srv48884403=79, srv241399702=62, srv1603428409=30, srv1936371345=49, srv1170056407=10, srv470589531=76, srv1312360475=15, srv1762512096=39, srv1789282461=40, srv320740777=67, srv488144053=78, srv802653840=92, srv327936480=70, srv783535854=91, srv884980484=96, srv989793695=99, srv1046122213=2, srv1158823477=9, srv473360763=77, srv1910763743=47, srv1728007067=38, srv893107185=97, srv1875247146=43, srv1485873399=22, srv2140156471=60, srv1875748893=44, srv759865138=88, srv11323319=7, srv1093739828=4, srv1503321609=24, srv1609662628=31, srv1993633753=53, srv1899957551=45, srv1455398647=20, srv1031589611=1, srv1580391217=27, srv118979206=12, srv711316236=86, srv323776586=68, srv903433874=98, srv1516085936=25, srv1098395931=5, srv1251238167=13, srv2022072653=55, srv1696716232=35, srv1580549134=28, srv2120488107=59, srv1573559226=26, srv770103462=89, srv723398121=87, srv1315256775=16, srv854153801=93, srv2043087176=56, srv1924892417=48, srv710286501=85, srv1439329535=18, srv1580563185=29, srv374153856=71, srv1093078777=3, srv291700199=64, srv1117093423=6, srv1623681351=32, srv1985094867=52, srv1332042352=17, srv30379575=65, srv309343592=66, srv466039233=75, srv19483478=50, srv1869700516=42, srv673608427=84, srv2104696355=58, srv1631266655=33, srv1260333602=14, srv384864453=73, srv1178455450=11, srv1999813323=54, srv443972030=74, srv1002813108=0, srv1443247829=19, srv873658994=95, srv1456570808=21, srv2102953440=57, srv32570759=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:39,469 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:39,470 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-05 15:53:39,471 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-05 15:53:39,472 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-05 15:53:39,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-05 15:53:39,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-05 15:53:39,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-05 15:53:39,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-05 15:53:39,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-05 15:53:39,609 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.387757500312559E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0016412198797244363); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:39,610 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1098395931:83 , srv1848738972:83 , srv384864453:83 , srv1002813108:84 , srv1031589611:84 , srv1046122213:84 , srv1093078777:84 , srv1093739828:84 , srv1117093423:84 , srv11323319:84 , srv1149260961:84 , srv1158823477:84 , srv1170056407:84 , srv1178455450:84 , srv118979206:84 , srv1251238167:84 , srv1260333602:84 , srv1312360475:84 , srv1315256775:84 , srv1332042352:84 , srv1439329535:84 , srv1443247829:84 , srv1455398647:84 , srv1456570808:84 , srv1485873399:84 , srv1490271233:84 , srv1503321609:84 , srv1516085936:84 , srv1573559226:84 , srv1580391217:84 , srv1580549134:84 , srv1580563185:84 , srv1603428409:84 , srv1609662628:84 , srv1623681351:84 , srv1631266655:84 , srv1634636808:84 , srv1696716232:84 , srv1706147715:84 , srv1713440546:84 , srv1728007067:84 , srv1762512096:84 , srv1869700516:84 , srv1875748893:84 , srv1899957551:84 , srv1909672333:84 , srv1910763743:84 , srv1924892417:84 , srv1936371345:84 , srv19483478:84 , srv1974630146:84 , srv1985094867:84 , srv1993633753:84 , srv1999813323:84 , srv2022072653:84 , srv2043087176:84 , srv2102953440:84 , srv2104696355:84 , srv2120488107:84 , srv2140156471:84 , srv239641880:84 , srv241399702:84 , srv255597364:84 , srv30379575:84 , srv309343592:84 , srv320740777:84 , srv323776586:84 , srv32570759:84 , srv327936480:84 , srv374153856:84 , srv374755388:84 , srv443972030:84 , srv466039233:84 , srv470589531:84 , srv473360763:84 , srv488144053:84 , srv48884403:84 , srv545681940:84 , srv602273411:84 , srv631432974:84 , srv636431025:84 , srv710286501:84 , srv711316236:84 , srv723398121:84 , srv759865138:84 , srv770103462:84 , srv780449553:84 , srv783535854:84 , srv802653840:84 , srv854153801:84 , srv861595971:84 , srv873658994:84 , srv884980484:84 , srv893107185:84 , srv903433874:84 , srv989793695:84 , srv1875247146:85 , srv291700199:85 , srv673608427:85 , srv1789282461:684 }
2023-05-05 15:53:39,649 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=366 (was 337) - SystemLoadAverage LEAK? -, ProcessCount=179 (was 179), AvailableMemoryMB=5778 (was 6637)
2023-05-05 15:53:39,666 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=366, ProcessCount=179, AvailableMemoryMB=5776
2023-05-05 15:53:39,666 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-05 15:53:39,667 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-05 15:53:39,677 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1289273432:84 , srv1302638733:84 , srv1341808810:84 , srv1532617954:84 , srv1599853395:84 , srv1616829289:84 , srv1700865236:84 , srv1728519123:84 , srv1864158908:84 , srv1948112116:84 , srv1949982971:84 , srv1980426619:84 , srv2022296903:84 , srv2037368735:84 , srv2056210585:84 , srv2136823339:84 , srv278199164:84 , srv417204289:84 , srv481943701:84 , srv521079882:84 , srv628636925:84 , srv637523162:84 , srv649534823:84 , srv745754600:84 , srv817512300:84 , srv878052404:84 , srv879245074:84 , srv915800547:84 , srv995703469:84 , srv616237897:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-05 15:53:39,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:53:39,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv649534823=23, srv1341808810=2, srv417204289=17, srv1864158908=8, srv1948112116=9, srv915800547=28, srv1980426619=11, srv2056210585=14, srv278199164=16, srv2022296903=12, srv878052404=26, srv1532617954=3, srv616237897=20, srv1289273432=0, srv879245074=27, srv1616829289=5, srv1728519123=7, srv995703469=29, srv1302638733=1, srv2037368735=13, srv817512300=25, srv1599853395=4, srv481943701=18, srv637523162=22, srv1700865236=6, srv745754600=24, srv1949982971=10, srv628636925=21, srv521079882=19, srv2136823339=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:53:39,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:53:39,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:53:39,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:53:39,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:53:39,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:53:39,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:53:39,694 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-05 15:53:39,714 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1146685133719179, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:53:39,714 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912554360005643, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1146685133719179, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-05 15:55:39,697 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5908005 different iterations.  Found a solution that moves 1872 regions; Going from a computed imbalance of 0.9912554360005643 to a new imbalance of 4.3693413878060563E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6933333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:55:39,703 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1289273432:90 , srv1302638733:90 , srv1341808810:90 , srv1532617954:90 , srv1599853395:90 , srv1616829289:90 , srv1700865236:90 , srv1728519123:90 , srv1864158908:90 , srv1948112116:90 , srv1949982971:90 , srv1980426619:90 , srv2022296903:90 , srv2037368735:90 , srv2056210585:90 , srv2136823339:90 , srv278199164:90 , srv417204289:90 , srv481943701:90 , srv521079882:90 , srv616237897:90 , srv628636925:90 , srv637523162:90 , srv649534823:90 , srv745754600:90 , srv817512300:90 , srv878052404:90 , srv879245074:90 , srv915800547:90 , srv995703469:90 }
2023-05-05 15:55:39,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-05 15:55:39,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv649534823=23, srv1341808810=2, srv417204289=17, srv1864158908=8, srv1948112116=9, srv915800547=28, srv1980426619=11, srv2056210585=14, srv278199164=16, srv2022296903=12, srv878052404=26, srv1532617954=3, srv616237897=20, srv1289273432=0, srv879245074=27, srv1616829289=5, srv1728519123=7, srv995703469=29, srv1302638733=1, srv2037368735=13, srv817512300=25, srv1599853395=4, srv481943701=18, srv637523162=22, srv1700865236=6, srv745754600=24, srv1949982971=10, srv628636925=21, srv521079882=19, srv2136823339=15} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-05 15:55:39,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-05 15:55:39,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-05 15:55:39,721 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-05 15:55:39,722 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1289273432:90 , srv1302638733:90 , srv1341808810:90 , srv1532617954:90 , srv1599853395:90 , srv1616829289:90 , srv1700865236:90 , srv1728519123:90 , srv1864158908:90 , srv1948112116:90 , srv1949982971:90 , srv1980426619:90 , srv2022296903:90 , srv2037368735:90 , srv2056210585:90 , srv2136823339:90 , srv278199164:90 , srv417204289:90 , srv481943701:90 , srv521079882:90 , srv616237897:90 , srv628636925:90 , srv637523162:90 , srv649534823:90 , srv745754600:90 , srv817512300:90 , srv878052404:90 , srv879245074:90 , srv915800547:90 , srv995703469:90 }
2023-05-05 15:55:39,735 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=290 (was 366), ProcessCount=179 (was 179), AvailableMemoryMB=5737 (was 5776)