2023-04-25 02:55:44,029 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-25 02:55:44,423 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-25 02:55:44,547 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-25 02:55:44,621 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-25 02:55:44,629 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-25 02:55:44,646 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=415, ProcessCount=170, AvailableMemoryMB=5042
2023-04-25 02:55:44,649 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-25 02:55:44,649 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-25 02:55:44,836 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000123677:84 , srv1060508395:84 , srv1064269555:84 , srv1132918648:84 , srv114280902:84 , srv1197914786:84 , srv1211761875:84 , srv1219470302:84 , srv1252705572:84 , srv1258054790:84 , srv1259023531:84 , srv1271852219:84 , srv1275347911:84 , srv1297101014:84 , srv1314637172:84 , srv1335797363:84 , srv1345326195:84 , srv1382335164:84 , srv1400563936:84 , srv1455403571:84 , srv1459237723:84 , srv1461505074:84 , srv1486719754:84 , srv1521401016:84 , srv1529198639:84 , srv1530948344:84 , srv1550968945:84 , srv156068003:84 , srv1562847640:84 , srv1569695965:84 , srv1590817009:84 , srv159471138:84 , srv1632151474:84 , srv1644067734:84 , srv1694552931:84 , srv1718430914:84 , srv1745049902:84 , srv1758760788:84 , srv1778468244:84 , srv1787905588:84 , srv1788561943:84 , srv1790015040:84 , srv1794648285:84 , srv1798102425:84 , srv1841369713:84 , srv1853067605:84 , srv1905733348:84 , srv200234136:84 , srv2018216708:84 , srv2022836601:84 , srv202466709:84 , srv2045193534:84 , srv2050877484:84 , srv2072139669:84 , srv2098904786:84 , srv2129611175:84 , srv240214757:84 , srv240507265:84 , srv26325466:84 , srv267185732:84 , srv299669379:84 , srv316704797:84 , srv321053912:84 , srv330990267:84 , srv352061736:84 , srv376666822:84 , srv387892872:84 , srv409400950:84 , srv432492787:84 , srv458831202:84 , srv512361361:84 , srv514163396:84 , srv527760306:84 , srv542960942:84 , srv557107036:84 , srv583939202:84 , srv584900108:84 , srv604384841:84 , srv6190807:84 , srv670606658:84 , srv691068428:84 , srv707220623:84 , srv717601333:84 , srv739763008:84 , srv773024557:84 , srv778990460:84 , srv79782617:84 , srv801756449:84 , srv802165804:84 , srv804976735:84 , srv817857132:84 , srv819755463:84 , srv859119378:84 , srv860603353:84 , srv924568104:84 , srv932544810:84 , srv960333182:84 , srv961284608:84 , srv961783805:84 , srv1648095344:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-25 02:55:44,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:44,861 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv200234136=48, srv1335797363=15, srv1590817009=30, srv670606658=80, srv2022836601=50, srv299669379=61, srv458831202=70, srv860603353=94, srv1718430914=36, srv159471138=31, srv514163396=72, srv604384841=78, srv1648095344=34, srv1060508395=1, srv1798102425=44, srv409400950=68, srv961284608=98, srv717601333=83, srv1745049902=37, srv1550968945=26, srv1455403571=19, srv1790015040=42, srv1905733348=47, srv1562847640=28, srv583939202=76, srv924568104=95, srv1486719754=22, srv156068003=27, srv1569695965=29, srv1271852219=11, srv1521401016=23, srv1794648285=43, srv1853067605=46, srv1345326195=16, srv1778468244=39, srv1259023531=10, srv527760306=73, srv1314637172=14, srv240214757=57, srv387892872=67, srv1788561943=41, srv691068428=81, srv1000123677=0, srv817857132=91, srv432492787=69, srv512361361=71, srv1758760788=38, srv2045193534=52, srv778990460=86, srv819755463=92, srv202466709=51, srv802165804=89, srv1064269555=2, srv961783805=99, srv1644067734=33, srv240507265=58, srv79782617=87, srv1694552931=35, srv2018216708=49, srv1632151474=32, srv1787905588=40, srv1459237723=20, srv1530948344=25, srv801756449=88, srv2098904786=55, srv859119378=93, srv352061736=65, srv1258054790=9, srv542960942=74, srv2129611175=56, srv1382335164=17, srv376666822=66, srv739763008=84, srv330990267=64, srv1461505074=21, srv2072139669=54, srv321053912=63, srv6190807=79, srv960333182=97, srv557107036=75, srv1219470302=7, srv1275347911=12, srv773024557=85, srv584900108=77, srv1297101014=13, srv316704797=62, srv1529198639=24, srv267185732=60, srv26325466=59, srv2050877484=53, srv707220623=82, srv1841369713=45, srv1252705572=8, srv932544810=96, srv1400563936=18, srv114280902=4, srv1197914786=5, srv1132918648=3, srv804976735=90, srv1211761875=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:45,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:45,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:45,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:45,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-25 02:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-25 02:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-25 02:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-25 02:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-25 02:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-25 02:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-25 02:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-25 02:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-25 02:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-25 02:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-25 02:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-25 02:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-25 02:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-25 02:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-25 02:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-25 02:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-25 02:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-25 02:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-25 02:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-25 02:55:45,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-25 02:55:45,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-25 02:55:45,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-25 02:55:45,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-25 02:55:45,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-25 02:55:45,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-25 02:55:45,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-25 02:55:45,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-25 02:55:45,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-25 02:55:45,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-25 02:55:45,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-25 02:55:45,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-25 02:55:45,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-25 02:55:45,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-25 02:55:45,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-25 02:55:45,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-25 02:55:45,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-25 02:55:45,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-25 02:55:45,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-25 02:55:45,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-25 02:55:45,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-25 02:55:45,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-25 02:55:45,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-25 02:55:45,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-25 02:55:45,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-25 02:55:45,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-25 02:55:45,360 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:45,360 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-25 02:55:45,361 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-25 02:55:45,822 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 758 ms to try 5000 different iterations.  Found a solution that moves 4689 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17667178450812251. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04095170356884037, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05723129279879038, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.521, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04095170356884037, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:45,857 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv114280902:81 , srv1000123677:82 , srv1314637172:82 , srv1562847640:82 , srv1569695965:82 , srv1798102425:82 , srv352061736:82 , srv514163396:82 , srv1459237723:83 , srv409400950:83 , srv859119378:83 , srv1455403571:84 , srv1787905588:84 , srv2022836601:84 , srv2050877484:84 , srv2098904786:84 , srv316704797:84 , srv330990267:84 , srv512361361:84 , srv961284608:84 , srv156068003:85 , srv159471138:85 , srv1694552931:85 , srv1778468244:85 , srv1841369713:85 , srv240507265:85 , srv267185732:85 , srv299669379:85 , srv376666822:85 , srv387892872:85 , srv432492787:85 , srv6190807:85 , srv691068428:85 , srv707220623:85 , srv924568104:85 , srv961783805:85 , srv1132918648:86 , srv1211761875:86 , srv1259023531:86 , srv1382335164:86 , srv1521401016:86 , srv1529198639:86 , srv1590817009:86 , srv1644067734:86 , srv1788561943:86 , srv1790015040:86 , srv200234136:86 , srv202466709:86 , srv2045193534:86 , srv26325466:86 , srv527760306:86 , srv557107036:86 , srv583939202:86 , srv773024557:86 , srv79782617:86 , srv817857132:86 , srv932544810:86 , srv1060508395:87 , srv1197914786:87 , srv1271852219:87 , srv1297101014:87 , srv1335797363:87 , srv1718430914:87 , srv1745049902:87 , srv1794648285:87 , srv1853067605:87 , srv2018216708:87 , srv240214757:87 , srv458831202:87 , srv670606658:87 , srv739763008:87 , srv960333182:87 , srv1252705572:88 , srv1275347911:88 , srv1461505074:88 , srv2129611175:88 , srv542960942:88 , srv584900108:88 , srv604384841:88 , srv717601333:88 , srv778990460:88 , srv801756449:88 , srv802165804:88 , srv1219470302:89 , srv1345326195:89 , srv1400563936:89 , srv1530948344:89 , srv1905733348:89 , srv804976735:89 , srv819755463:89 , srv1064269555:90 , srv1258054790:90 , srv1632151474:90 , srv321053912:90 , srv2072139669:91 , srv860603353:91 , srv1758760788:92 , srv1550968945:93 , srv1486719754:97 , srv1648095344:454 }
2023-04-25 02:55:45,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:45,865 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv200234136=48, srv1335797363=15, srv1590817009=30, srv670606658=80, srv2022836601=50, srv299669379=61, srv458831202=70, srv860603353=94, srv1718430914=36, srv159471138=31, srv514163396=72, srv604384841=78, srv1648095344=34, srv1060508395=1, srv1798102425=44, srv409400950=68, srv961284608=98, srv717601333=83, srv1745049902=37, srv1550968945=26, srv1455403571=19, srv1790015040=42, srv1905733348=47, srv1562847640=28, srv583939202=76, srv924568104=95, srv1486719754=22, srv156068003=27, srv1569695965=29, srv1271852219=11, srv1521401016=23, srv1794648285=43, srv1853067605=46, srv1345326195=16, srv1778468244=39, srv1259023531=10, srv527760306=73, srv1314637172=14, srv240214757=57, srv387892872=67, srv1788561943=41, srv691068428=81, srv1000123677=0, srv817857132=91, srv432492787=69, srv512361361=71, srv1758760788=38, srv2045193534=52, srv778990460=86, srv819755463=92, srv202466709=51, srv802165804=89, srv1064269555=2, srv961783805=99, srv1644067734=33, srv240507265=58, srv79782617=87, srv1694552931=35, srv2018216708=49, srv1632151474=32, srv1787905588=40, srv1459237723=20, srv1530948344=25, srv801756449=88, srv2098904786=55, srv859119378=93, srv352061736=65, srv1258054790=9, srv542960942=74, srv2129611175=56, srv1382335164=17, srv376666822=66, srv739763008=84, srv330990267=64, srv1461505074=21, srv2072139669=54, srv321053912=63, srv6190807=79, srv960333182=97, srv557107036=75, srv1219470302=7, srv1275347911=12, srv773024557=85, srv584900108=77, srv1297101014=13, srv316704797=62, srv1529198639=24, srv267185732=60, srv26325466=59, srv2050877484=53, srv707220623=82, srv1841369713=45, srv1252705572=8, srv932544810=96, srv1400563936=18, srv114280902=4, srv1197914786=5, srv1132918648=3, srv804976735=90, srv1211761875=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:45,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:45,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:45,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:45,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:45,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-25 02:55:45,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-25 02:55:45,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-25 02:55:45,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-25 02:55:45,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-25 02:55:45,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-25 02:55:45,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-25 02:55:45,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-25 02:55:45,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-25 02:55:45,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-25 02:55:45,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-25 02:55:45,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-25 02:55:45,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-25 02:55:45,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-25 02:55:45,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-25 02:55:45,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-25 02:55:45,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-25 02:55:45,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-25 02:55:45,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-25 02:55:45,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-25 02:55:45,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-25 02:55:45,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-25 02:55:45,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-25 02:55:45,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-25 02:55:45,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-25 02:55:45,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-25 02:55:45,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-25 02:55:45,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-25 02:55:45,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-25 02:55:45,911 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-25 02:55:45,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-25 02:55:45,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-25 02:55:45,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-25 02:55:45,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-25 02:55:45,912 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-25 02:55:45,913 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-25 02:55:45,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-25 02:55:45,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-25 02:55:45,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-25 02:55:45,914 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-25 02:55:46,122 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04095170356884037, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05723129279879038, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04095170356884037, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:46,123 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-25 02:55:46,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17663895142836702, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04095170356884037, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05723129279879038, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04095170356884037, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17141666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24283333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-25 02:55:46,533 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 551 ms to try 5000 different iterations.  Found a solution that moves 2689 regions; Going from a computed imbalance of 0.17663895142836702 to a new imbalance of 0.015894796630692366. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019812690656779295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032376849469856, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29877777777777775, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019812690656779295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.050333333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:46,550 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1486719754:76 , srv1258054790:81 , srv1550968945:81 , srv804976735:81 , srv1400563936:82 , srv1632151474:82 , srv1758760788:82 , srv2072139669:82 , srv321053912:82 , srv778990460:82 , srv802165804:82 , srv1064269555:83 , srv1345326195:83 , srv1530948344:83 , srv1905733348:83 , srv542960942:83 , srv860603353:83 , srv1219470302:84 , srv1271852219:84 , srv1275347911:84 , srv2045193534:84 , srv2129611175:84 , srv584900108:84 , srv773024557:84 , srv819755463:84 , srv1197914786:85 , srv1211761875:85 , srv1252705572:85 , srv1335797363:85 , srv1382335164:85 , srv1529198639:85 , srv1718430914:85 , srv1745049902:85 , srv1788561943:85 , srv1790015040:85 , srv1853067605:85 , srv2018216708:85 , srv202466709:85 , srv240507265:85 , srv376666822:85 , srv604384841:85 , srv670606658:85 , srv79782617:85 , srv801756449:85 , srv1060508395:86 , srv1259023531:86 , srv1461505074:86 , srv1590817009:86 , srv1778468244:86 , srv200234136:86 , srv240214757:86 , srv387892872:86 , srv512361361:86 , srv691068428:86 , srv707220623:86 , srv717601333:86 , srv817857132:86 , srv924568104:86 , srv1297101014:87 , srv1459237723:87 , srv1521401016:87 , srv156068003:87 , srv1644067734:87 , srv1794648285:87 , srv2050877484:87 , srv2098904786:87 , srv26325466:87 , srv267185732:87 , srv316704797:87 , srv432492787:87 , srv557107036:87 , srv859119378:87 , srv932544810:87 , srv159471138:88 , srv2022836601:88 , srv299669379:88 , srv409400950:88 , srv458831202:88 , srv527760306:88 , srv583939202:88 , srv6190807:88 , srv739763008:88 , srv960333182:88 , srv961284608:88 , srv1314637172:89 , srv1455403571:89 , srv1569695965:89 , srv1694552931:89 , srv1787905588:89 , srv1841369713:89 , srv1000123677:90 , srv1562847640:90 , srv330990267:90 , srv514163396:90 , srv961783805:90 , srv1132918648:91 , srv1798102425:91 , srv352061736:91 , srv114280902:92 , srv1648095344:496 }
2023-04-25 02:55:46,556 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:46,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv200234136=48, srv1335797363=15, srv1590817009=30, srv670606658=80, srv2022836601=50, srv299669379=61, srv458831202=70, srv860603353=94, srv1718430914=36, srv159471138=31, srv514163396=72, srv604384841=78, srv1648095344=34, srv1060508395=1, srv1798102425=44, srv409400950=68, srv961284608=98, srv717601333=83, srv1745049902=37, srv1550968945=26, srv1455403571=19, srv1790015040=42, srv1905733348=47, srv1562847640=28, srv583939202=76, srv924568104=95, srv1486719754=22, srv156068003=27, srv1569695965=29, srv1271852219=11, srv1521401016=23, srv1794648285=43, srv1853067605=46, srv1345326195=16, srv1778468244=39, srv1259023531=10, srv527760306=73, srv1314637172=14, srv240214757=57, srv387892872=67, srv1788561943=41, srv691068428=81, srv1000123677=0, srv817857132=91, srv432492787=69, srv512361361=71, srv1758760788=38, srv2045193534=52, srv778990460=86, srv819755463=92, srv202466709=51, srv802165804=89, srv1064269555=2, srv961783805=99, srv1644067734=33, srv240507265=58, srv79782617=87, srv1694552931=35, srv2018216708=49, srv1632151474=32, srv1787905588=40, srv1459237723=20, srv1530948344=25, srv801756449=88, srv2098904786=55, srv859119378=93, srv352061736=65, srv1258054790=9, srv542960942=74, srv2129611175=56, srv1382335164=17, srv376666822=66, srv739763008=84, srv330990267=64, srv1461505074=21, srv2072139669=54, srv321053912=63, srv6190807=79, srv960333182=97, srv557107036=75, srv1219470302=7, srv1275347911=12, srv773024557=85, srv584900108=77, srv1297101014=13, srv316704797=62, srv1529198639=24, srv267185732=60, srv26325466=59, srv2050877484=53, srv707220623=82, srv1841369713=45, srv1252705572=8, srv932544810=96, srv1400563936=18, srv114280902=4, srv1197914786=5, srv1132918648=3, srv804976735=90, srv1211761875=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:46,580 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:46,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:46,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:46,581 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:46,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:46,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:46,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:46,582 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:46,583 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:46,584 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-25 02:55:46,585 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-25 02:55:46,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-25 02:55:46,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-25 02:55:46,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-25 02:55:46,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-25 02:55:46,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-25 02:55:46,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-25 02:55:46,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-25 02:55:46,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-25 02:55:46,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-25 02:55:46,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-25 02:55:46,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-25 02:55:46,734 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019812690656779295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032376849469856, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019812690656779295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.050333333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:46,735 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-25 02:55:46,735 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015875967850256772, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019812690656779295); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032376849469856, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019812690656779295); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012333333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.050333333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-25 02:55:47,146 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 533 ms to try 5000 different iterations.  Found a solution that moves 1017 regions; Going from a computed imbalance of 0.015875967850256772 to a new imbalance of 8.673228144937284E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004984074901000224); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00743093905363617); MoveCostFunction : (multiplier=7.0, imbalance=0.113, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004984074901000224); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:47,149 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1132918648:81 , srv960333182:82 , srv458831202:83 , srv717601333:83 , srv860603353:83 , srv961783805:83 , srv1060508395:84 , srv1064269555:84 , srv114280902:84 , srv1219470302:84 , srv1252705572:84 , srv1297101014:84 , srv1461505074:84 , srv1486719754:84 , srv1521401016:84 , srv1550968945:84 , srv159471138:84 , srv1644067734:84 , srv1694552931:84 , srv1787905588:84 , srv1794648285:84 , srv1841369713:84 , srv240214757:84 , srv299669379:84 , srv330990267:84 , srv352061736:84 , srv527760306:84 , srv583939202:84 , srv604384841:84 , srv739763008:84 , srv801756449:84 , srv819755463:84 , srv1259023531:85 , srv1275347911:85 , srv1335797363:85 , srv1345326195:85 , srv1455403571:85 , srv1530948344:85 , srv156068003:85 , srv1562847640:85 , srv1718430914:85 , srv1758760788:85 , srv1905733348:85 , srv2022836601:85 , srv2072139669:85 , srv26325466:85 , srv432492787:85 , srv584900108:85 , srv6190807:85 , srv817857132:85 , srv932544810:85 , srv961284608:85 , srv1000123677:86 , srv1197914786:86 , srv1211761875:86 , srv1271852219:86 , srv1314637172:86 , srv1382335164:86 , srv1400563936:86 , srv1529198639:86 , srv1590817009:86 , srv1632151474:86 , srv1745049902:86 , srv1778468244:86 , srv1790015040:86 , srv1798102425:86 , srv1853067605:86 , srv200234136:86 , srv2018216708:86 , srv202466709:86 , srv2050877484:86 , srv2129611175:86 , srv267185732:86 , srv316704797:86 , srv321053912:86 , srv387892872:86 , srv514163396:86 , srv557107036:86 , srv670606658:86 , srv79782617:86 , srv924568104:86 , srv1258054790:87 , srv1569695965:87 , srv1788561943:87 , srv2045193534:87 , srv2098904786:87 , srv240507265:87 , srv542960942:87 , srv691068428:87 , srv707220623:87 , srv859119378:87 , srv1459237723:88 , srv376666822:88 , srv409400950:88 , srv512361361:88 , srv773024557:88 , srv778990460:88 , srv804976735:88 , srv802165804:89 , srv1648095344:552 }
2023-04-25 02:55:47,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:47,156 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv200234136=48, srv1335797363=15, srv1590817009=30, srv670606658=80, srv2022836601=50, srv299669379=61, srv458831202=70, srv860603353=94, srv1718430914=36, srv159471138=31, srv514163396=72, srv604384841=78, srv1648095344=34, srv1060508395=1, srv1798102425=44, srv409400950=68, srv961284608=98, srv717601333=83, srv1745049902=37, srv1550968945=26, srv1455403571=19, srv1790015040=42, srv1905733348=47, srv1562847640=28, srv583939202=76, srv924568104=95, srv1486719754=22, srv156068003=27, srv1569695965=29, srv1271852219=11, srv1521401016=23, srv1794648285=43, srv1853067605=46, srv1345326195=16, srv1778468244=39, srv1259023531=10, srv527760306=73, srv1314637172=14, srv240214757=57, srv387892872=67, srv1788561943=41, srv691068428=81, srv1000123677=0, srv817857132=91, srv432492787=69, srv512361361=71, srv1758760788=38, srv2045193534=52, srv778990460=86, srv819755463=92, srv202466709=51, srv802165804=89, srv1064269555=2, srv961783805=99, srv1644067734=33, srv240507265=58, srv79782617=87, srv1694552931=35, srv2018216708=49, srv1632151474=32, srv1787905588=40, srv1459237723=20, srv1530948344=25, srv801756449=88, srv2098904786=55, srv859119378=93, srv352061736=65, srv1258054790=9, srv542960942=74, srv2129611175=56, srv1382335164=17, srv376666822=66, srv739763008=84, srv330990267=64, srv1461505074=21, srv2072139669=54, srv321053912=63, srv6190807=79, srv960333182=97, srv557107036=75, srv1219470302=7, srv1275347911=12, srv773024557=85, srv584900108=77, srv1297101014=13, srv316704797=62, srv1529198639=24, srv267185732=60, srv26325466=59, srv2050877484=53, srv707220623=82, srv1841369713=45, srv1252705572=8, srv932544810=96, srv1400563936=18, srv114280902=4, srv1197914786=5, srv1132918648=3, srv804976735=90, srv1211761875=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:47,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:47,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:47,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:47,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:47,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-25 02:55:47,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-25 02:55:47,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-25 02:55:47,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-25 02:55:47,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-25 02:55:47,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-25 02:55:47,187 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-25 02:55:47,188 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-25 02:55:47,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-25 02:55:47,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-25 02:55:47,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-25 02:55:47,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-25 02:55:47,324 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004984074901000224); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00743093905363617); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004984074901000224); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:47,324 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-25 02:55:47,325 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.602016282895638E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004984074901000224); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00743093905363617); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004984074901000224); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00475); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-25 02:55:47,691 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 482 ms to try 5000 different iterations.  Found a solution that moves 228 regions; Going from a computed imbalance of 8.602016282895638E-4 to a new imbalance of 2.9009147924554502E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.025333333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:47,692 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1590817009:83 , srv1758760788:83 , srv1798102425:83 , srv2072139669:83 , srv376666822:83 , srv409400950:83 , srv458831202:83 , srv542960942:83 , srv557107036:83 , srv739763008:83 , srv802165804:83 , srv804976735:83 , srv1000123677:84 , srv1064269555:84 , srv1197914786:84 , srv1258054790:84 , srv1400563936:84 , srv1459237723:84 , srv1461505074:84 , srv1486719754:84 , srv1550968945:84 , srv1569695965:84 , srv1632151474:84 , srv1694552931:84 , srv1745049902:84 , srv1788561943:84 , srv1794648285:84 , srv1841369713:84 , srv1853067605:84 , srv200234136:84 , srv2018216708:84 , srv2098904786:84 , srv2129611175:84 , srv240214757:84 , srv26325466:84 , srv267185732:84 , srv321053912:84 , srv514163396:84 , srv527760306:84 , srv583939202:84 , srv584900108:84 , srv6190807:84 , srv691068428:84 , srv707220623:84 , srv773024557:84 , srv778990460:84 , srv932544810:84 , srv961284608:84 , srv961783805:84 , srv1060508395:85 , srv1132918648:85 , srv1211761875:85 , srv1259023531:85 , srv1275347911:85 , srv1297101014:85 , srv1314637172:85 , srv1335797363:85 , srv1345326195:85 , srv1382335164:85 , srv1455403571:85 , srv1521401016:85 , srv1529198639:85 , srv1530948344:85 , srv156068003:85 , srv1562847640:85 , srv159471138:85 , srv1644067734:85 , srv1718430914:85 , srv1778468244:85 , srv1790015040:85 , srv1905733348:85 , srv2022836601:85 , srv202466709:85 , srv2045193534:85 , srv2050877484:85 , srv240507265:85 , srv299669379:85 , srv316704797:85 , srv330990267:85 , srv352061736:85 , srv387892872:85 , srv432492787:85 , srv512361361:85 , srv604384841:85 , srv670606658:85 , srv79782617:85 , srv801756449:85 , srv817857132:85 , srv819755463:85 , srv859119378:85 , srv860603353:85 , srv924568104:85 , srv960333182:85 , srv114280902:86 , srv1219470302:86 , srv1252705572:86 , srv1271852219:86 , srv1787905588:86 , srv717601333:86 , srv1648095344:640 }
2023-04-25 02:55:47,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:47,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv200234136=48, srv1335797363=15, srv1590817009=30, srv670606658=80, srv2022836601=50, srv299669379=61, srv458831202=70, srv860603353=94, srv1718430914=36, srv159471138=31, srv514163396=72, srv604384841=78, srv1648095344=34, srv1060508395=1, srv1798102425=44, srv409400950=68, srv961284608=98, srv717601333=83, srv1745049902=37, srv1550968945=26, srv1455403571=19, srv1790015040=42, srv1905733348=47, srv1562847640=28, srv583939202=76, srv924568104=95, srv1486719754=22, srv156068003=27, srv1569695965=29, srv1271852219=11, srv1521401016=23, srv1794648285=43, srv1853067605=46, srv1345326195=16, srv1778468244=39, srv1259023531=10, srv527760306=73, srv1314637172=14, srv240214757=57, srv387892872=67, srv1788561943=41, srv691068428=81, srv1000123677=0, srv817857132=91, srv432492787=69, srv512361361=71, srv1758760788=38, srv2045193534=52, srv778990460=86, srv819755463=92, srv202466709=51, srv802165804=89, srv1064269555=2, srv961783805=99, srv1644067734=33, srv240507265=58, srv79782617=87, srv1694552931=35, srv2018216708=49, srv1632151474=32, srv1787905588=40, srv1459237723=20, srv1530948344=25, srv801756449=88, srv2098904786=55, srv859119378=93, srv352061736=65, srv1258054790=9, srv542960942=74, srv2129611175=56, srv1382335164=17, srv376666822=66, srv739763008=84, srv330990267=64, srv1461505074=21, srv2072139669=54, srv321053912=63, srv6190807=79, srv960333182=97, srv557107036=75, srv1219470302=7, srv1275347911=12, srv773024557=85, srv584900108=77, srv1297101014=13, srv316704797=62, srv1529198639=24, srv267185732=60, srv26325466=59, srv2050877484=53, srv707220623=82, srv1841369713=45, srv1252705572=8, srv932544810=96, srv1400563936=18, srv114280902=4, srv1197914786=5, srv1132918648=3, srv804976735=90, srv1211761875=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:47,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:47,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:47,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-25 02:55:47,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-25 02:55:47,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-25 02:55:47,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-25 02:55:47,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-25 02:55:47,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-25 02:55:47,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-25 02:55:47,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-25 02:55:47,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-25 02:55:47,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-25 02:55:47,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-25 02:55:47,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-25 02:55:47,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-25 02:55:47,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-25 02:55:47,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-25 02:55:47,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-25 02:55:47,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-25 02:55:47,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-25 02:55:47,862 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:47,862 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-25 02:55:47,862 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.7412657802086904E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-25 02:55:48,176 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 444 ms to try 5000 different iterations.  Found a solution that moves 37 regions; Going from a computed imbalance of 2.7412657802086904E-5 to a new imbalance of 6.657065209687686E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.004111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:48,176 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1132918648:83 , srv114280902:83 , srv1219470302:83 , srv1252705572:83 , srv1271852219:83 , srv1297101014:83 , srv1455403571:83 , srv1787905588:83 , srv330990267:83 , srv512361361:83 , srv670606658:83 , srv717601333:83 , srv1000123677:84 , srv1060508395:84 , srv1197914786:84 , srv1211761875:84 , srv1258054790:84 , srv1259023531:84 , srv1275347911:84 , srv1314637172:84 , srv1335797363:84 , srv1345326195:84 , srv1382335164:84 , srv1459237723:84 , srv1461505074:84 , srv1521401016:84 , srv1529198639:84 , srv1530948344:84 , srv1550968945:84 , srv156068003:84 , srv1562847640:84 , srv1569695965:84 , srv159471138:84 , srv1632151474:84 , srv1644067734:84 , srv1694552931:84 , srv1718430914:84 , srv1745049902:84 , srv1758760788:84 , srv1778468244:84 , srv1788561943:84 , srv1790015040:84 , srv1794648285:84 , srv1798102425:84 , srv1841369713:84 , srv1853067605:84 , srv1905733348:84 , srv200234136:84 , srv2018216708:84 , srv2022836601:84 , srv202466709:84 , srv2045193534:84 , srv2050877484:84 , srv2098904786:84 , srv2129611175:84 , srv240507265:84 , srv26325466:84 , srv267185732:84 , srv299669379:84 , srv316704797:84 , srv321053912:84 , srv352061736:84 , srv387892872:84 , srv409400950:84 , srv432492787:84 , srv514163396:84 , srv527760306:84 , srv557107036:84 , srv583939202:84 , srv604384841:84 , srv6190807:84 , srv691068428:84 , srv707220623:84 , srv739763008:84 , srv773024557:84 , srv778990460:84 , srv79782617:84 , srv801756449:84 , srv802165804:84 , srv817857132:84 , srv819755463:84 , srv859119378:84 , srv860603353:84 , srv924568104:84 , srv932544810:84 , srv960333182:84 , srv961783805:84 , srv1064269555:85 , srv1400563936:85 , srv1486719754:85 , srv1590817009:85 , srv2072139669:85 , srv240214757:85 , srv376666822:85 , srv458831202:85 , srv542960942:85 , srv584900108:85 , srv804976735:85 , srv961284608:85 , srv1648095344:684 }
2023-04-25 02:55:48,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:48,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv200234136=48, srv1335797363=15, srv1590817009=30, srv670606658=80, srv2022836601=50, srv299669379=61, srv458831202=70, srv860603353=94, srv1718430914=36, srv159471138=31, srv514163396=72, srv604384841=78, srv1648095344=34, srv1060508395=1, srv1798102425=44, srv409400950=68, srv961284608=98, srv717601333=83, srv1745049902=37, srv1550968945=26, srv1455403571=19, srv1790015040=42, srv1905733348=47, srv1562847640=28, srv583939202=76, srv924568104=95, srv1486719754=22, srv156068003=27, srv1569695965=29, srv1271852219=11, srv1521401016=23, srv1794648285=43, srv1853067605=46, srv1345326195=16, srv1778468244=39, srv1259023531=10, srv527760306=73, srv1314637172=14, srv240214757=57, srv387892872=67, srv1788561943=41, srv691068428=81, srv1000123677=0, srv817857132=91, srv432492787=69, srv512361361=71, srv1758760788=38, srv2045193534=52, srv778990460=86, srv819755463=92, srv202466709=51, srv802165804=89, srv1064269555=2, srv961783805=99, srv1644067734=33, srv240507265=58, srv79782617=87, srv1694552931=35, srv2018216708=49, srv1632151474=32, srv1787905588=40, srv1459237723=20, srv1530948344=25, srv801756449=88, srv2098904786=55, srv859119378=93, srv352061736=65, srv1258054790=9, srv542960942=74, srv2129611175=56, srv1382335164=17, srv376666822=66, srv739763008=84, srv330990267=64, srv1461505074=21, srv2072139669=54, srv321053912=63, srv6190807=79, srv960333182=97, srv557107036=75, srv1219470302=7, srv1275347911=12, srv773024557=85, srv584900108=77, srv1297101014=13, srv316704797=62, srv1529198639=24, srv267185732=60, srv26325466=59, srv2050877484=53, srv707220623=82, srv1841369713=45, srv1252705572=8, srv932544810=96, srv1400563936=18, srv114280902=4, srv1197914786=5, srv1132918648=3, srv804976735=90, srv1211761875=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:48,194 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:48,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-25 02:55:48,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-25 02:55:48,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-25 02:55:48,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-25 02:55:48,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-25 02:55:48,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-25 02:55:48,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-25 02:55:48,316 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:48,316 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1132918648:83 , srv114280902:83 , srv1219470302:83 , srv1252705572:83 , srv1271852219:83 , srv1297101014:83 , srv1455403571:83 , srv1787905588:83 , srv330990267:83 , srv512361361:83 , srv670606658:83 , srv717601333:83 , srv1000123677:84 , srv1060508395:84 , srv1197914786:84 , srv1211761875:84 , srv1258054790:84 , srv1259023531:84 , srv1275347911:84 , srv1314637172:84 , srv1335797363:84 , srv1345326195:84 , srv1382335164:84 , srv1459237723:84 , srv1461505074:84 , srv1521401016:84 , srv1529198639:84 , srv1530948344:84 , srv1550968945:84 , srv156068003:84 , srv1562847640:84 , srv1569695965:84 , srv159471138:84 , srv1632151474:84 , srv1644067734:84 , srv1694552931:84 , srv1718430914:84 , srv1745049902:84 , srv1758760788:84 , srv1778468244:84 , srv1788561943:84 , srv1790015040:84 , srv1794648285:84 , srv1798102425:84 , srv1841369713:84 , srv1853067605:84 , srv1905733348:84 , srv200234136:84 , srv2018216708:84 , srv2022836601:84 , srv202466709:84 , srv2045193534:84 , srv2050877484:84 , srv2098904786:84 , srv2129611175:84 , srv240507265:84 , srv26325466:84 , srv267185732:84 , srv299669379:84 , srv316704797:84 , srv321053912:84 , srv352061736:84 , srv387892872:84 , srv409400950:84 , srv432492787:84 , srv514163396:84 , srv527760306:84 , srv557107036:84 , srv583939202:84 , srv604384841:84 , srv6190807:84 , srv691068428:84 , srv707220623:84 , srv739763008:84 , srv773024557:84 , srv778990460:84 , srv79782617:84 , srv801756449:84 , srv802165804:84 , srv817857132:84 , srv819755463:84 , srv859119378:84 , srv860603353:84 , srv924568104:84 , srv932544810:84 , srv960333182:84 , srv961783805:84 , srv1064269555:85 , srv1400563936:85 , srv1486719754:85 , srv1590817009:85 , srv2072139669:85 , srv240214757:85 , srv376666822:85 , srv458831202:85 , srv542960942:85 , srv584900108:85 , srv804976735:85 , srv961284608:85 , srv1648095344:684 }
2023-04-25 02:55:48,350 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=454 (was 415) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=4580 (was 5042)
2023-04-25 02:55:48,375 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=262, MaxFileDescriptor=60000, SystemLoadAverage=454, ProcessCount=170, AvailableMemoryMB=4580
2023-04-25 02:55:48,375 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-25 02:55:48,378 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-25 02:55:48,392 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015147214:84 , srv1028521627:84 , srv1055054445:84 , srv1117801870:84 , srv1301394206:84 , srv1362865328:84 , srv1419982661:84 , srv1528737898:84 , srv1541655722:84 , srv1564225963:84 , srv186967151:84 , srv1876557950:84 , srv1968950537:84 , srv1970375131:84 , srv206299844:84 , srv280304506:84 , srv306556316:84 , srv42592947:84 , srv49204192:84 , srv493671614:84 , srv615717747:84 , srv644182554:84 , srv662329816:84 , srv690535356:84 , srv721242102:84 , srv792833850:84 , srv816194978:84 , srv898707898:84 , srv919986438:84 , srv859498630:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-25 02:55:48,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:55:48,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1419982661=6, srv1055054445=2, srv1015147214=0, srv1028521627=1, srv690535356=23, srv919986438=29, srv898707898=28, srv1528737898=7, srv1117801870=3, srv644182554=21, srv1876557950=11, srv615717747=20, srv1301394206=4, srv49204192=18, srv1362865328=5, srv721242102=24, srv1968950537=12, srv1541655722=8, srv186967151=10, srv493671614=19, srv816194978=26, srv1564225963=9, srv206299844=14, srv1970375131=13, srv792833850=25, srv859498630=27, srv662329816=22, srv280304506=15, srv306556316=16, srv42592947=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:55:48,408 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:55:48,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:55:48,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:55:48,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:55:48,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:55:48,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:55:48,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-25 02:55:48,433 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1468373125716569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:55:48,434 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912655722841512, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1468373125716569, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-25 02:57:48,415 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5522059 different iterations.  Found a solution that moves 1869 regions; Going from a computed imbalance of 0.9912655722841512 to a new imbalance of 4.362339238146111E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6922222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:57:48,421 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1015147214:90 , srv1028521627:90 , srv1055054445:90 , srv1117801870:90 , srv1301394206:90 , srv1362865328:90 , srv1419982661:90 , srv1528737898:90 , srv1541655722:90 , srv1564225963:90 , srv186967151:90 , srv1876557950:90 , srv1968950537:90 , srv1970375131:90 , srv206299844:90 , srv280304506:90 , srv306556316:90 , srv42592947:90 , srv49204192:90 , srv493671614:90 , srv615717747:90 , srv644182554:90 , srv662329816:90 , srv690535356:90 , srv721242102:90 , srv792833850:90 , srv816194978:90 , srv859498630:90 , srv898707898:90 , srv919986438:90 }
2023-04-25 02:57:48,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-25 02:57:48,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1419982661=6, srv1055054445=2, srv1015147214=0, srv1028521627=1, srv690535356=23, srv919986438=29, srv898707898=28, srv1528737898=7, srv1117801870=3, srv644182554=21, srv1876557950=11, srv615717747=20, srv1301394206=4, srv49204192=18, srv1362865328=5, srv721242102=24, srv1968950537=12, srv1541655722=8, srv186967151=10, srv493671614=19, srv816194978=26, srv1564225963=9, srv206299844=14, srv1970375131=13, srv792833850=25, srv859498630=27, srv662329816=22, srv280304506=15, srv306556316=16, srv42592947=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-25 02:57:48,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-25 02:57:48,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-25 02:57:48,438 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-25 02:57:48,438 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1015147214:90 , srv1028521627:90 , srv1055054445:90 , srv1117801870:90 , srv1301394206:90 , srv1362865328:90 , srv1419982661:90 , srv1528737898:90 , srv1541655722:90 , srv1564225963:90 , srv186967151:90 , srv1876557950:90 , srv1968950537:90 , srv1970375131:90 , srv206299844:90 , srv280304506:90 , srv306556316:90 , srv42592947:90 , srv49204192:90 , srv493671614:90 , srv615717747:90 , srv644182554:90 , srv662329816:90 , srv690535356:90 , srv721242102:90 , srv792833850:90 , srv816194978:90 , srv859498630:90 , srv898707898:90 , srv919986438:90 }
2023-04-25 02:57:48,450 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 262), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=313 (was 454), ProcessCount=170 (was 170), AvailableMemoryMB=5073 (was 4580) - AvailableMemoryMB LEAK? -