2023-05-02 04:53:34,312 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-02 04:53:34,665 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-02 04:53:34,785 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-02 04:53:34,860 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-02 04:53:34,868 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-02 04:53:34,882 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=364, ProcessCount=170, AvailableMemoryMB=15299
2023-05-02 04:53:34,884 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-02 04:53:34,884 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-02 04:53:35,017 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1007490165:84 , srv1013742829:84 , srv101808616:84 , srv1027409188:84 , srv1032943927:84 , srv1036438864:84 , srv1056378477:84 , srv1059467999:84 , srv1059752794:84 , srv1069398423:84 , srv1119231717:84 , srv1139556604:84 , srv1151104696:84 , srv1158744696:84 , srv1160504697:84 , srv1166453394:84 , srv1229620767:84 , srv1241168817:84 , srv1248661171:84 , srv1257104013:84 , srv1275285889:84 , srv1278837938:84 , srv1302851496:84 , srv1316048062:84 , srv1357872129:84 , srv1366547686:84 , srv1405447885:84 , srv1431397472:84 , srv1449193908:84 , srv1469038940:84 , srv1501635943:84 , srv1522744269:84 , srv1523990711:84 , srv1527240348:84 , srv1535434029:84 , srv1572622400:84 , srv1583688588:84 , srv1635696106:84 , srv165414029:84 , srv1668643543:84 , srv1676329674:84 , srv1687260068:84 , srv1687723100:84 , srv170395503:84 , srv173187534:84 , srv1732100387:84 , srv1828038225:84 , srv1836062411:84 , srv1893220005:84 , srv1893801459:84 , srv1909768901:84 , srv1935047350:84 , srv1951921540:84 , srv1953102596:84 , srv1973789297:84 , srv1982003047:84 , srv2074710305:84 , srv230311810:84 , srv233569680:84 , srv25138334:84 , srv252646562:84 , srv254087082:84 , srv273935748:84 , srv276923303:84 , srv305837780:84 , srv421290150:84 , srv441509276:84 , srv442180343:84 , srv445922620:84 , srv494970273:84 , srv540769852:84 , srv550750778:84 , srv552476172:84 , srv563696579:84 , srv564449797:84 , srv6042317:84 , srv626537302:84 , srv640643885:84 , srv658501110:84 , srv696019376:84 , srv697059412:84 , srv699638258:84 , srv705300729:84 , srv715845126:84 , srv736879892:84 , srv743883402:84 , srv746855036:84 , srv749879176:84 , srv75766568:84 , srv780347242:84 , srv781515729:84 , srv798062745:84 , srv866590272:84 , srv882061363:84 , srv912047045:84 , srv912794843:84 , srv934230079:84 , srv943999169:84 , srv95961135:84 , srv1184850917:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-02 04:53:35,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:35,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1449193908=29, srv705300729=83, srv743883402=86, srv1059467999=7, srv1302851496=23, srv626537302=77, srv1732100387=46, srv912047045=95, srv1501635943=31, srv780347242=90, srv445922620=69, srv165414029=39, srv1119231717=10, srv1357872129=25, srv1687723100=43, srv798062745=92, srv696019376=80, srv1151104696=12, srv1676329674=41, srv230311810=58, srv1229620767=17, srv95961135=99, srv1069398423=9, srv550750778=72, srv564449797=75, srv1535434029=35, srv1953102596=54, srv1893801459=50, srv254087082=62, srv1007490165=0, srv1635696106=38, srv1951921540=53, srv1405447885=27, srv563696579=74, srv170395503=44, srv715845126=84, srv276923303=64, srv1027409188=3, srv1036438864=5, srv1158744696=13, srv1973789297=55, srv442180343=68, srv1013742829=1, srv1032943927=4, srv736879892=85, srv1668643543=40, srv233569680=59, srv540769852=71, srv1469038940=30, srv943999169=98, srv1572622400=36, srv1583688588=37, srv75766568=89, srv1248661171=19, srv1257104013=20, srv640643885=78, srv912794843=96, srv1523990711=33, srv781515729=91, srv1431397472=28, srv1184850917=16, srv25138334=60, srv6042317=76, srv1836062411=48, srv882061363=94, srv1241168817=18, srv1160504697=14, srv1893220005=49, srv1935047350=52, srv1278837938=22, srv1275285889=21, srv1527240348=34, srv1139556604=11, srv749879176=88, srv421290150=66, srv101808616=2, srv552476172=73, srv697059412=81, srv658501110=79, srv1166453394=15, srv441509276=67, srv746855036=87, srv305837780=65, srv866590272=93, srv494970273=70, srv1687260068=42, srv1366547686=26, srv1909768901=51, srv2074710305=57, srv1522744269=32, srv1316048062=24, srv1056378477=6, srv273935748=63, srv1982003047=56, srv934230079=97, srv1828038225=47, srv699638258=82, srv1059752794=8, srv173187534=45, srv252646562=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:35,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:35,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:35,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:35,135 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:35,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:35,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:35,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:35,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:35,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:35,136 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:35,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:35,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:35,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:35,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:35,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:35,137 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:35,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:35,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:35,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:35,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:35,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:35,138 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:35,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-02 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-02 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-02 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-02 04:53:35,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-02 04:53:35,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-02 04:53:35,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-02 04:53:35,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-02 04:53:35,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-02 04:53:35,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-02 04:53:35,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-02 04:53:35,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-02 04:53:35,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-02 04:53:35,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-02 04:53:35,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-02 04:53:35,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-02 04:53:35,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-02 04:53:35,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-02 04:53:35,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-02 04:53:35,455 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:35,455 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-02 04:53:35,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-02 04:53:35,812 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 629 ms to try 5000 different iterations.  Found a solution that moves 4672 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1844868743883778. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04160425554029572, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056046317225336116, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5191111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04160425554029572, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17991666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:35,823 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1013742829:82 , srv1501635943:82 , srv1583688588:82 , srv25138334:82 , srv1036438864:83 , srv1166453394:83 , srv1357872129:83 , srv254087082:83 , srv276923303:83 , srv746855036:83 , srv780347242:83 , srv1056378477:84 , srv1160504697:84 , srv1229620767:84 , srv1257104013:84 , srv1405447885:84 , srv1449193908:84 , srv1469038940:84 , srv1523990711:84 , srv1527240348:84 , srv1535434029:84 , srv1687723100:84 , srv252646562:84 , srv563696579:84 , srv705300729:84 , srv912047045:84 , srv1032943927:85 , srv1241168817:85 , srv1316048062:85 , srv1676329674:85 , srv173187534:85 , srv1893801459:85 , srv1982003047:85 , srv233569680:85 , srv305837780:85 , srv441509276:85 , srv442180343:85 , srv494970273:85 , srv696019376:85 , srv743883402:85 , srv934230079:85 , srv1059752794:86 , srv1158744696:86 , srv1302851496:86 , srv1366547686:86 , srv1687260068:86 , srv1836062411:86 , srv1953102596:86 , srv540769852:86 , srv564449797:86 , srv715845126:86 , srv1069398423:87 , srv1139556604:87 , srv1151104696:87 , srv1248661171:87 , srv1522744269:87 , srv1572622400:87 , srv165414029:87 , srv1732100387:87 , srv1828038225:87 , srv1893220005:87 , srv1935047350:87 , srv1951921540:87 , srv626537302:87 , srv697059412:87 , srv749879176:87 , srv75766568:87 , srv866590272:87 , srv882061363:87 , srv1027409188:88 , srv1119231717:88 , srv1278837938:88 , srv1431397472:88 , srv273935748:88 , srv445922620:88 , srv550750778:88 , srv640643885:88 , srv658501110:88 , srv736879892:88 , srv943999169:88 , srv95961135:88 , srv1059467999:89 , srv1275285889:89 , srv1635696106:89 , srv1668643543:89 , srv1909768901:89 , srv230311810:89 , srv421290150:89 , srv552476172:89 , srv781515729:89 , srv798062745:89 , srv912794843:89 , srv1007490165:90 , srv101808616:90 , srv699638258:90 , srv170395503:91 , srv2074710305:91 , srv6042317:91 , srv1973789297:92 , srv1184850917:460 }
2023-05-02 04:53:35,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:35,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1449193908=29, srv705300729=83, srv743883402=86, srv1059467999=7, srv1302851496=23, srv626537302=77, srv1732100387=46, srv912047045=95, srv1501635943=31, srv780347242=90, srv445922620=69, srv165414029=39, srv1119231717=10, srv1357872129=25, srv1687723100=43, srv798062745=92, srv696019376=80, srv1151104696=12, srv1676329674=41, srv230311810=58, srv1229620767=17, srv95961135=99, srv1069398423=9, srv550750778=72, srv564449797=75, srv1535434029=35, srv1953102596=54, srv1893801459=50, srv254087082=62, srv1007490165=0, srv1635696106=38, srv1951921540=53, srv1405447885=27, srv563696579=74, srv170395503=44, srv715845126=84, srv276923303=64, srv1027409188=3, srv1036438864=5, srv1158744696=13, srv1973789297=55, srv442180343=68, srv1013742829=1, srv1032943927=4, srv736879892=85, srv1668643543=40, srv233569680=59, srv540769852=71, srv1469038940=30, srv943999169=98, srv1572622400=36, srv1583688588=37, srv75766568=89, srv1248661171=19, srv1257104013=20, srv640643885=78, srv912794843=96, srv1523990711=33, srv781515729=91, srv1431397472=28, srv1184850917=16, srv25138334=60, srv6042317=76, srv1836062411=48, srv882061363=94, srv1241168817=18, srv1160504697=14, srv1893220005=49, srv1935047350=52, srv1278837938=22, srv1275285889=21, srv1527240348=34, srv1139556604=11, srv749879176=88, srv421290150=66, srv101808616=2, srv552476172=73, srv697059412=81, srv658501110=79, srv1166453394=15, srv441509276=67, srv746855036=87, srv305837780=65, srv866590272=93, srv494970273=70, srv1687260068=42, srv1366547686=26, srv1909768901=51, srv2074710305=57, srv1522744269=32, srv1316048062=24, srv1056378477=6, srv273935748=63, srv1982003047=56, srv934230079=97, srv1828038225=47, srv699638258=82, srv1059752794=8, srv173187534=45, srv252646562=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:35,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:35,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:35,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:35,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:35,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:35,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-02 04:53:35,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-02 04:53:35,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-02 04:53:35,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-02 04:53:35,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-02 04:53:35,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-02 04:53:35,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-02 04:53:35,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-02 04:53:35,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-02 04:53:35,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-02 04:53:35,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-02 04:53:35,998 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04160425554029572, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056046317225336116, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04160425554029572, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17991666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:35,999 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-02 04:53:35,999 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18445416034516654, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04160425554029572, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.056046317225336116, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04160425554029572, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17991666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-02 04:53:36,277 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 394 ms to try 5000 different iterations.  Found a solution that moves 2573 regions; Going from a computed imbalance of 0.18445416034516654 to a new imbalance of 0.01726464848269508. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027427420120849803, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037060935050889365, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2858888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.027427420120849803, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:36,288 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1973789297:79 , srv421290150:80 , srv6042317:80 , srv699638258:80 , srv1007490165:81 , srv1275285889:81 , srv1431397472:81 , srv1668643543:81 , srv170395503:81 , srv2074710305:81 , srv798062745:81 , srv101808616:82 , srv550750778:82 , srv552476172:82 , srv781515729:82 , srv912794843:82 , srv1027409188:83 , srv1119231717:83 , srv1635696106:83 , srv1828038225:83 , srv1935047350:83 , srv1953102596:83 , srv273935748:83 , srv445922620:83 , srv626537302:83 , srv640643885:83 , srv658501110:83 , srv943999169:83 , srv1032943927:84 , srv1059467999:84 , srv1151104696:84 , srv1248661171:84 , srv1687260068:84 , srv1732100387:84 , srv697059412:84 , srv866590272:84 , srv1069398423:85 , srv1241168817:85 , srv1257104013:85 , srv1278837938:85 , srv1522744269:85 , srv1572622400:85 , srv1836062411:85 , srv1893801459:85 , srv1909768901:85 , srv1951921540:85 , srv230311810:85 , srv305837780:85 , srv441509276:85 , srv442180343:85 , srv540769852:85 , srv736879892:85 , srv749879176:85 , srv1139556604:86 , srv1366547686:86 , srv1405447885:86 , srv173187534:86 , srv705300729:86 , srv743883402:86 , srv75766568:86 , srv95961135:86 , srv1059752794:87 , srv1158744696:87 , srv1160504697:87 , srv1166453394:87 , srv1449193908:87 , srv1523990711:87 , srv165414029:87 , srv1893220005:87 , srv1982003047:87 , srv252646562:87 , srv494970273:87 , srv563696579:87 , srv715845126:87 , srv882061363:87 , srv912047045:87 , srv934230079:87 , srv1036438864:88 , srv1056378477:88 , srv1229620767:88 , srv1302851496:88 , srv1676329674:88 , srv1687723100:88 , srv233569680:88 , srv564449797:88 , srv746855036:88 , srv780347242:88 , srv1316048062:89 , srv1357872129:89 , srv1469038940:89 , srv1501635943:89 , srv1527240348:89 , srv1535434029:89 , srv1583688588:89 , srv25138334:89 , srv696019376:89 , srv1013742829:90 , srv254087082:91 , srv276923303:91 , srv1184850917:558 }
2023-05-02 04:53:36,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:36,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1449193908=29, srv705300729=83, srv743883402=86, srv1059467999=7, srv1302851496=23, srv626537302=77, srv1732100387=46, srv912047045=95, srv1501635943=31, srv780347242=90, srv445922620=69, srv165414029=39, srv1119231717=10, srv1357872129=25, srv1687723100=43, srv798062745=92, srv696019376=80, srv1151104696=12, srv1676329674=41, srv230311810=58, srv1229620767=17, srv95961135=99, srv1069398423=9, srv550750778=72, srv564449797=75, srv1535434029=35, srv1953102596=54, srv1893801459=50, srv254087082=62, srv1007490165=0, srv1635696106=38, srv1951921540=53, srv1405447885=27, srv563696579=74, srv170395503=44, srv715845126=84, srv276923303=64, srv1027409188=3, srv1036438864=5, srv1158744696=13, srv1973789297=55, srv442180343=68, srv1013742829=1, srv1032943927=4, srv736879892=85, srv1668643543=40, srv233569680=59, srv540769852=71, srv1469038940=30, srv943999169=98, srv1572622400=36, srv1583688588=37, srv75766568=89, srv1248661171=19, srv1257104013=20, srv640643885=78, srv912794843=96, srv1523990711=33, srv781515729=91, srv1431397472=28, srv1184850917=16, srv25138334=60, srv6042317=76, srv1836062411=48, srv882061363=94, srv1241168817=18, srv1160504697=14, srv1893220005=49, srv1935047350=52, srv1278837938=22, srv1275285889=21, srv1527240348=34, srv1139556604=11, srv749879176=88, srv421290150=66, srv101808616=2, srv552476172=73, srv697059412=81, srv658501110=79, srv1166453394=15, srv441509276=67, srv746855036=87, srv305837780=65, srv866590272=93, srv494970273=70, srv1687260068=42, srv1366547686=26, srv1909768901=51, srv2074710305=57, srv1522744269=32, srv1316048062=24, srv1056378477=6, srv273935748=63, srv1982003047=56, srv934230079=97, srv1828038225=47, srv699638258=82, srv1059752794=8, srv173187534=45, srv252646562=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:36,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:36,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:36,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:36,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:36,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:36,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:36,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:36,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-02 04:53:36,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-02 04:53:36,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-02 04:53:36,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-02 04:53:36,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-02 04:53:36,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-02 04:53:36,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-02 04:53:36,513 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027427420120849803, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037060935050889365, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.027427420120849803, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:36,514 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-02 04:53:36,514 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01724663195162004, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027427420120849803, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.037060935050889365, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.027427420120849803, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.014166666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04658333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-02 04:53:36,782 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 421 ms to try 5000 different iterations.  Found a solution that moves 1014 regions; Going from a computed imbalance of 0.01724663195162004 to a new imbalance of 0.001305821335470498. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006545856356568484); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008178699991918044); MoveCostFunction : (multiplier=7.0, imbalance=0.11266666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006545856356568484); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:36,785 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv95961135:82 , srv1316048062:83 , srv165414029:83 , srv1893220005:83 , srv564449797:83 , srv1059467999:84 , srv1059752794:84 , srv1139556604:84 , srv1158744696:84 , srv1302851496:84 , srv1676329674:84 , srv1909768901:84 , srv1982003047:84 , srv254087082:84 , srv276923303:84 , srv696019376:84 , srv715845126:84 , srv736879892:84 , srv882061363:84 , srv1013742829:85 , srv1027409188:85 , srv1056378477:85 , srv1229620767:85 , srv1278837938:85 , srv1357872129:85 , srv1469038940:85 , srv1523990711:85 , srv1527240348:85 , srv1535434029:85 , srv170395503:85 , srv230311810:85 , srv233569680:85 , srv749879176:85 , srv75766568:85 , srv1007490165:86 , srv101808616:86 , srv1069398423:86 , srv1119231717:86 , srv1151104696:86 , srv1248661171:86 , srv1501635943:86 , srv1522744269:86 , srv1572622400:86 , srv1583688588:86 , srv1635696106:86 , srv1687723100:86 , srv173187534:86 , srv1732100387:86 , srv1836062411:86 , srv1951921540:86 , srv2074710305:86 , srv25138334:86 , srv252646562:86 , srv273935748:86 , srv445922620:86 , srv494970273:86 , srv540769852:86 , srv552476172:86 , srv640643885:86 , srv697059412:86 , srv743883402:86 , srv746855036:86 , srv781515729:86 , srv912047045:86 , srv934230079:86 , srv943999169:86 , srv1036438864:87 , srv1160504697:87 , srv1166453394:87 , srv1241168817:87 , srv1366547686:87 , srv1405447885:87 , srv1449193908:87 , srv1668643543:87 , srv1828038225:87 , srv1935047350:87 , srv1973789297:87 , srv441509276:87 , srv442180343:87 , srv563696579:87 , srv6042317:87 , srv658501110:87 , srv705300729:87 , srv780347242:87 , srv798062745:87 , srv866590272:87 , srv912794843:87 , srv1032943927:88 , srv1257104013:88 , srv1275285889:88 , srv1431397472:88 , srv1687260068:88 , srv1893801459:88 , srv305837780:88 , srv421290150:88 , srv550750778:88 , srv626537302:88 , srv1953102596:89 , srv699638258:89 , srv1184850917:498 }
2023-05-02 04:53:36,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:36,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1449193908=29, srv705300729=83, srv743883402=86, srv1059467999=7, srv1302851496=23, srv626537302=77, srv1732100387=46, srv912047045=95, srv1501635943=31, srv780347242=90, srv445922620=69, srv165414029=39, srv1119231717=10, srv1357872129=25, srv1687723100=43, srv798062745=92, srv696019376=80, srv1151104696=12, srv1676329674=41, srv230311810=58, srv1229620767=17, srv95961135=99, srv1069398423=9, srv550750778=72, srv564449797=75, srv1535434029=35, srv1953102596=54, srv1893801459=50, srv254087082=62, srv1007490165=0, srv1635696106=38, srv1951921540=53, srv1405447885=27, srv563696579=74, srv170395503=44, srv715845126=84, srv276923303=64, srv1027409188=3, srv1036438864=5, srv1158744696=13, srv1973789297=55, srv442180343=68, srv1013742829=1, srv1032943927=4, srv736879892=85, srv1668643543=40, srv233569680=59, srv540769852=71, srv1469038940=30, srv943999169=98, srv1572622400=36, srv1583688588=37, srv75766568=89, srv1248661171=19, srv1257104013=20, srv640643885=78, srv912794843=96, srv1523990711=33, srv781515729=91, srv1431397472=28, srv1184850917=16, srv25138334=60, srv6042317=76, srv1836062411=48, srv882061363=94, srv1241168817=18, srv1160504697=14, srv1893220005=49, srv1935047350=52, srv1278837938=22, srv1275285889=21, srv1527240348=34, srv1139556604=11, srv749879176=88, srv421290150=66, srv101808616=2, srv552476172=73, srv697059412=81, srv658501110=79, srv1166453394=15, srv441509276=67, srv746855036=87, srv305837780=65, srv866590272=93, srv494970273=70, srv1687260068=42, srv1366547686=26, srv1909768901=51, srv2074710305=57, srv1522744269=32, srv1316048062=24, srv1056378477=6, srv273935748=63, srv1982003047=56, srv934230079=97, srv1828038225=47, srv699638258=82, srv1059752794=8, srv173187534=45, srv252646562=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:36,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:36,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:36,811 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:36,812 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-02 04:53:36,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-02 04:53:36,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-02 04:53:36,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-02 04:53:36,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-02 04:53:36,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-02 04:53:36,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-02 04:53:36,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-02 04:53:36,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-02 04:53:36,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-02 04:53:36,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-02 04:53:36,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-02 04:53:36,977 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006545856356568484); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008178699991918044); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006545856356568484); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:36,977 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-02 04:53:36,977 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0012987211557153131, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006545856356568484); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008178699991918044); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006545856356568484); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333334E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-02 04:53:37,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 583 ms to try 5000 different iterations.  Found a solution that moves 286 regions; Going from a computed imbalance of 0.0012987211557153131 to a new imbalance of 2.8931747180222985E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.03177777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:37,423 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1366547686:83 , srv1572622400:83 , srv230311810:83 , srv6042317:83 , srv699638258:83 , srv101808616:84 , srv1032943927:84 , srv1036438864:84 , srv1069398423:84 , srv1151104696:84 , srv1160504697:84 , srv1257104013:84 , srv1275285889:84 , srv1278837938:84 , srv1302851496:84 , srv1449193908:84 , srv1469038940:84 , srv1522744269:84 , srv1535434029:84 , srv1635696106:84 , srv1893220005:84 , srv1893801459:84 , srv1909768901:84 , srv1951921540:84 , srv1953102596:84 , srv1973789297:84 , srv2074710305:84 , srv233569680:84 , srv254087082:84 , srv276923303:84 , srv305837780:84 , srv421290150:84 , srv494970273:84 , srv550750778:84 , srv563696579:84 , srv626537302:84 , srv658501110:84 , srv696019376:84 , srv75766568:84 , srv780347242:84 , srv866590272:84 , srv882061363:84 , srv912794843:84 , srv934230079:84 , srv1007490165:85 , srv1013742829:85 , srv1056378477:85 , srv1059467999:85 , srv1059752794:85 , srv1119231717:85 , srv1139556604:85 , srv1158744696:85 , srv1229620767:85 , srv1241168817:85 , srv1248661171:85 , srv1316048062:85 , srv1357872129:85 , srv1405447885:85 , srv1431397472:85 , srv1501635943:85 , srv1527240348:85 , srv1583688588:85 , srv1668643543:85 , srv1676329674:85 , srv1687260068:85 , srv1687723100:85 , srv170395503:85 , srv173187534:85 , srv1732100387:85 , srv1828038225:85 , srv1836062411:85 , srv1935047350:85 , srv25138334:85 , srv252646562:85 , srv273935748:85 , srv441509276:85 , srv442180343:85 , srv445922620:85 , srv552476172:85 , srv564449797:85 , srv640643885:85 , srv697059412:85 , srv705300729:85 , srv715845126:85 , srv736879892:85 , srv743883402:85 , srv746855036:85 , srv749879176:85 , srv781515729:85 , srv798062745:85 , srv912047045:85 , srv1027409188:86 , srv1166453394:86 , srv165414029:86 , srv1982003047:86 , srv540769852:86 , srv943999169:86 , srv95961135:86 , srv1523990711:87 , srv1184850917:625 }
2023-05-02 04:53:37,428 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:37,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1449193908=29, srv705300729=83, srv743883402=86, srv1059467999=7, srv1302851496=23, srv626537302=77, srv1732100387=46, srv912047045=95, srv1501635943=31, srv780347242=90, srv445922620=69, srv165414029=39, srv1119231717=10, srv1357872129=25, srv1687723100=43, srv798062745=92, srv696019376=80, srv1151104696=12, srv1676329674=41, srv230311810=58, srv1229620767=17, srv95961135=99, srv1069398423=9, srv550750778=72, srv564449797=75, srv1535434029=35, srv1953102596=54, srv1893801459=50, srv254087082=62, srv1007490165=0, srv1635696106=38, srv1951921540=53, srv1405447885=27, srv563696579=74, srv170395503=44, srv715845126=84, srv276923303=64, srv1027409188=3, srv1036438864=5, srv1158744696=13, srv1973789297=55, srv442180343=68, srv1013742829=1, srv1032943927=4, srv736879892=85, srv1668643543=40, srv233569680=59, srv540769852=71, srv1469038940=30, srv943999169=98, srv1572622400=36, srv1583688588=37, srv75766568=89, srv1248661171=19, srv1257104013=20, srv640643885=78, srv912794843=96, srv1523990711=33, srv781515729=91, srv1431397472=28, srv1184850917=16, srv25138334=60, srv6042317=76, srv1836062411=48, srv882061363=94, srv1241168817=18, srv1160504697=14, srv1893220005=49, srv1935047350=52, srv1278837938=22, srv1275285889=21, srv1527240348=34, srv1139556604=11, srv749879176=88, srv421290150=66, srv101808616=2, srv552476172=73, srv697059412=81, srv658501110=79, srv1166453394=15, srv441509276=67, srv746855036=87, srv305837780=65, srv866590272=93, srv494970273=70, srv1687260068=42, srv1366547686=26, srv1909768901=51, srv2074710305=57, srv1522744269=32, srv1316048062=24, srv1056378477=6, srv273935748=63, srv1982003047=56, srv934230079=97, srv1828038225=47, srv699638258=82, srv1059752794=8, srv173187534=45, srv252646562=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:37,447 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:37,448 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-02 04:53:37,449 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-02 04:53:37,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-02 04:53:37,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-02 04:53:37,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-02 04:53:37,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-02 04:53:37,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-02 04:53:37,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-02 04:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-02 04:53:37,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-02 04:53:37,578 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:37,578 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-02 04:53:37,578 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.6929132377478546E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0021711298242630987); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-02 04:53:37,823 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 353 ms to try 5000 different iterations.  Found a solution that moves 28 regions; Going from a computed imbalance of 2.6929132377478546E-5 to a new imbalance of 8.175750307519066E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.003111111111111111); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:37,824 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1166453394:83 , srv1523990711:83 , srv1527240348:83 , srv165414029:83 , srv1687260068:83 , srv1687723100:83 , srv540769852:83 , srv943999169:83 , srv1007490165:84 , srv1013742829:84 , srv101808616:84 , srv1027409188:84 , srv1036438864:84 , srv1056378477:84 , srv1059467999:84 , srv1059752794:84 , srv1069398423:84 , srv1119231717:84 , srv1139556604:84 , srv1158744696:84 , srv1160504697:84 , srv1229620767:84 , srv1241168817:84 , srv1248661171:84 , srv1275285889:84 , srv1278837938:84 , srv1302851496:84 , srv1316048062:84 , srv1357872129:84 , srv1366547686:84 , srv1405447885:84 , srv1431397472:84 , srv1449193908:84 , srv1469038940:84 , srv1501635943:84 , srv1522744269:84 , srv1535434029:84 , srv1583688588:84 , srv1635696106:84 , srv1668643543:84 , srv1676329674:84 , srv170395503:84 , srv173187534:84 , srv1732100387:84 , srv1828038225:84 , srv1836062411:84 , srv1893801459:84 , srv1909768901:84 , srv1935047350:84 , srv1951921540:84 , srv1953102596:84 , srv1973789297:84 , srv1982003047:84 , srv2074710305:84 , srv230311810:84 , srv233569680:84 , srv25138334:84 , srv252646562:84 , srv254087082:84 , srv273935748:84 , srv276923303:84 , srv305837780:84 , srv441509276:84 , srv442180343:84 , srv445922620:84 , srv494970273:84 , srv550750778:84 , srv552476172:84 , srv563696579:84 , srv564449797:84 , srv626537302:84 , srv640643885:84 , srv658501110:84 , srv696019376:84 , srv697059412:84 , srv699638258:84 , srv705300729:84 , srv715845126:84 , srv736879892:84 , srv743883402:84 , srv746855036:84 , srv749879176:84 , srv75766568:84 , srv780347242:84 , srv781515729:84 , srv798062745:84 , srv866590272:84 , srv882061363:84 , srv912047045:84 , srv912794843:84 , srv934230079:84 , srv95961135:84 , srv1032943927:85 , srv1151104696:85 , srv1257104013:85 , srv1572622400:85 , srv1893220005:85 , srv421290150:85 , srv6042317:85 , srv1184850917:685 }
2023-05-02 04:53:37,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:37,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1449193908=29, srv705300729=83, srv743883402=86, srv1059467999=7, srv1302851496=23, srv626537302=77, srv1732100387=46, srv912047045=95, srv1501635943=31, srv780347242=90, srv445922620=69, srv165414029=39, srv1119231717=10, srv1357872129=25, srv1687723100=43, srv798062745=92, srv696019376=80, srv1151104696=12, srv1676329674=41, srv230311810=58, srv1229620767=17, srv95961135=99, srv1069398423=9, srv550750778=72, srv564449797=75, srv1535434029=35, srv1953102596=54, srv1893801459=50, srv254087082=62, srv1007490165=0, srv1635696106=38, srv1951921540=53, srv1405447885=27, srv563696579=74, srv170395503=44, srv715845126=84, srv276923303=64, srv1027409188=3, srv1036438864=5, srv1158744696=13, srv1973789297=55, srv442180343=68, srv1013742829=1, srv1032943927=4, srv736879892=85, srv1668643543=40, srv233569680=59, srv540769852=71, srv1469038940=30, srv943999169=98, srv1572622400=36, srv1583688588=37, srv75766568=89, srv1248661171=19, srv1257104013=20, srv640643885=78, srv912794843=96, srv1523990711=33, srv781515729=91, srv1431397472=28, srv1184850917=16, srv25138334=60, srv6042317=76, srv1836062411=48, srv882061363=94, srv1241168817=18, srv1160504697=14, srv1893220005=49, srv1935047350=52, srv1278837938=22, srv1275285889=21, srv1527240348=34, srv1139556604=11, srv749879176=88, srv421290150=66, srv101808616=2, srv552476172=73, srv697059412=81, srv658501110=79, srv1166453394=15, srv441509276=67, srv746855036=87, srv305837780=65, srv866590272=93, srv494970273=70, srv1687260068=42, srv1366547686=26, srv1909768901=51, srv2074710305=57, srv1522744269=32, srv1316048062=24, srv1056378477=6, srv273935748=63, srv1982003047=56, srv934230079=97, srv1828038225=47, srv699638258=82, srv1059752794=8, srv173187534=45, srv252646562=61} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:37,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:37,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:37,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:37,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:37,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:37,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:37,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-02 04:53:37,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-02 04:53:37,842 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-02 04:53:37,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-02 04:53:37,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-02 04:53:37,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-02 04:53:37,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-02 04:53:37,959 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.979690117040589E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017727200782610348); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:37,959 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1166453394:83 , srv1523990711:83 , srv1527240348:83 , srv165414029:83 , srv1687260068:83 , srv1687723100:83 , srv540769852:83 , srv943999169:83 , srv1007490165:84 , srv1013742829:84 , srv101808616:84 , srv1027409188:84 , srv1036438864:84 , srv1056378477:84 , srv1059467999:84 , srv1059752794:84 , srv1069398423:84 , srv1119231717:84 , srv1139556604:84 , srv1158744696:84 , srv1160504697:84 , srv1229620767:84 , srv1241168817:84 , srv1248661171:84 , srv1275285889:84 , srv1278837938:84 , srv1302851496:84 , srv1316048062:84 , srv1357872129:84 , srv1366547686:84 , srv1405447885:84 , srv1431397472:84 , srv1449193908:84 , srv1469038940:84 , srv1501635943:84 , srv1522744269:84 , srv1535434029:84 , srv1583688588:84 , srv1635696106:84 , srv1668643543:84 , srv1676329674:84 , srv170395503:84 , srv173187534:84 , srv1732100387:84 , srv1828038225:84 , srv1836062411:84 , srv1893801459:84 , srv1909768901:84 , srv1935047350:84 , srv1951921540:84 , srv1953102596:84 , srv1973789297:84 , srv1982003047:84 , srv2074710305:84 , srv230311810:84 , srv233569680:84 , srv25138334:84 , srv252646562:84 , srv254087082:84 , srv273935748:84 , srv276923303:84 , srv305837780:84 , srv441509276:84 , srv442180343:84 , srv445922620:84 , srv494970273:84 , srv550750778:84 , srv552476172:84 , srv563696579:84 , srv564449797:84 , srv626537302:84 , srv640643885:84 , srv658501110:84 , srv696019376:84 , srv697059412:84 , srv699638258:84 , srv705300729:84 , srv715845126:84 , srv736879892:84 , srv743883402:84 , srv746855036:84 , srv749879176:84 , srv75766568:84 , srv780347242:84 , srv781515729:84 , srv798062745:84 , srv866590272:84 , srv882061363:84 , srv912047045:84 , srv912794843:84 , srv934230079:84 , srv95961135:84 , srv1032943927:85 , srv1151104696:85 , srv1257104013:85 , srv1572622400:85 , srv1893220005:85 , srv421290150:85 , srv6042317:85 , srv1184850917:685 }
2023-05-02 04:53:37,998 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=364 (was 364), ProcessCount=170 (was 170), AvailableMemoryMB=14442 (was 15299)
2023-05-02 04:53:38,009 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=364, ProcessCount=170, AvailableMemoryMB=14441
2023-05-02 04:53:38,010 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-02 04:53:38,011 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-02 04:53:38,022 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1095384932:84 , srv1145454415:84 , srv1194223018:84 , srv1201918991:84 , srv1256491260:84 , srv1277308708:84 , srv132454452:84 , srv1405416095:84 , srv1565798405:84 , srv1626656512:84 , srv1629962537:84 , srv1656617456:84 , srv1689920970:84 , srv1730849066:84 , srv1751323285:84 , srv2036660003:84 , srv2079764375:84 , srv239447893:84 , srv255209058:84 , srv260512016:84 , srv342530484:84 , srv394645503:84 , srv510155588:84 , srv514465135:84 , srv521979918:84 , srv743392710:84 , srv839582531:84 , srv862815492:84 , srv866697439:84 , srv854752366:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-02 04:53:38,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:53:38,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv255209058=18, srv1751323285=14, srv862815492=28, srv1145454415=1, srv2079764375=16, srv510155588=22, srv866697439=29, srv239447893=17, srv342530484=20, srv514465135=23, srv1629962537=10, srv260512016=19, srv1689920970=12, srv521979918=24, srv1565798405=8, srv1201918991=3, srv743392710=25, srv854752366=27, srv1256491260=4, srv1626656512=9, srv2036660003=15, srv1194223018=2, srv1405416095=7, srv1656617456=11, srv1277308708=5, srv839582531=26, srv394645503=21, srv1095384932=0, srv132454452=6, srv1730849066=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:53:38,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:53:38,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:53:38,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:53:38,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:53:38,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:53:38,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:53:38,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:53:38,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-02 04:53:38,063 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.154179634692332, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:53:38,063 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912678858258767, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.154179634692332, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-02 04:55:38,043 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6971385 different iterations.  Found a solution that moves 1855 regions; Going from a computed imbalance of 0.9912678858258767 to a new imbalance of 4.329662539733032E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.687037037037037, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:55:38,050 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1095384932:90 , srv1145454415:90 , srv1194223018:90 , srv1201918991:90 , srv1256491260:90 , srv1277308708:90 , srv132454452:90 , srv1405416095:90 , srv1565798405:90 , srv1626656512:90 , srv1629962537:90 , srv1656617456:90 , srv1689920970:90 , srv1730849066:90 , srv1751323285:90 , srv2036660003:90 , srv2079764375:90 , srv239447893:90 , srv255209058:90 , srv260512016:90 , srv342530484:90 , srv394645503:90 , srv510155588:90 , srv514465135:90 , srv521979918:90 , srv743392710:90 , srv839582531:90 , srv854752366:90 , srv862815492:90 , srv866697439:90 }
2023-05-02 04:55:38,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-02 04:55:38,051 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv255209058=18, srv1751323285=14, srv862815492=28, srv1145454415=1, srv2079764375=16, srv510155588=22, srv866697439=29, srv239447893=17, srv342530484=20, srv514465135=23, srv1629962537=10, srv260512016=19, srv1689920970=12, srv521979918=24, srv1565798405=8, srv1201918991=3, srv743392710=25, srv854752366=27, srv1256491260=4, srv1626656512=9, srv2036660003=15, srv1194223018=2, srv1405416095=7, srv1656617456=11, srv1277308708=5, srv839582531=26, srv394645503=21, srv1095384932=0, srv132454452=6, srv1730849066=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-02 04:55:38,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-02 04:55:38,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-02 04:55:38,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-02 04:55:38,054 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-02 04:55:38,055 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-02 04:55:38,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-02 04:55:38,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-02 04:55:38,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-02 04:55:38,073 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-02 04:55:38,073 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1095384932:90 , srv1145454415:90 , srv1194223018:90 , srv1201918991:90 , srv1256491260:90 , srv1277308708:90 , srv132454452:90 , srv1405416095:90 , srv1565798405:90 , srv1626656512:90 , srv1629962537:90 , srv1656617456:90 , srv1689920970:90 , srv1730849066:90 , srv1751323285:90 , srv2036660003:90 , srv2079764375:90 , srv239447893:90 , srv255209058:90 , srv260512016:90 , srv342530484:90 , srv394645503:90 , srv510155588:90 , srv514465135:90 , srv521979918:90 , srv743392710:90 , srv839582531:90 , srv854752366:90 , srv862815492:90 , srv866697439:90 }
2023-05-02 04:55:38,088 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=185 (was 364), ProcessCount=170 (was 170), AvailableMemoryMB=14236 (was 14441)