2023-02-08 23:55:52,501 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-08 23:55:52,808 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-08 23:55:52,919 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-08 23:55:52,984 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-08 23:55:52,991 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-08 23:55:53,004 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=168, AvailableMemoryMB=3878
2023-02-08 23:55:53,006 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-08 23:55:53,007 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-08 23:55:53,136 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1012442395:84 , srv1018151695:84 , srv1043954974:84 , srv105367128:84 , srv1061561561:84 , srv1075849561:84 , srv1079184834:84 , srv1086825642:84 , srv1107132613:84 , srv1116081893:84 , srv1134965045:84 , srv1148897379:84 , srv1154036226:84 , srv1166178002:84 , srv1172362935:84 , srv119809865:84 , srv1238463825:84 , srv1282308414:84 , srv1322964938:84 , srv1359585843:84 , srv1362463723:84 , srv1551558945:84 , srv1576918960:84 , srv1587197620:84 , srv1725252724:84 , srv1729094057:84 , srv1750544602:84 , srv1773119410:84 , srv1782023470:84 , srv1784963943:84 , srv1788413199:84 , srv1794870019:84 , srv1802903568:84 , srv182272299:84 , srv1827449449:84 , srv1829083632:84 , srv1858398074:84 , srv1872036560:84 , srv1873123471:84 , srv1874961726:84 , srv189391870:84 , srv1922765778:84 , srv1946986734:84 , srv1959481559:84 , srv1990091384:84 , srv1999290689:84 , srv2047975824:84 , srv2076156349:84 , srv2086317914:84 , srv2097631809:84 , srv2132219180:84 , srv2133168291:84 , srv2133478015:84 , srv247039581:84 , srv250278493:84 , srv257466230:84 , srv316246632:84 , srv322700195:84 , srv356567053:84 , srv407916686:84 , srv427209349:84 , srv430639069:84 , srv465468185:84 , srv473516507:84 , srv480171503:84 , srv482679894:84 , srv493215069:84 , srv514400583:84 , srv556682182:84 , srv556929091:84 , srv557114887:84 , srv591369232:84 , srv591480086:84 , srv614584620:84 , srv644390649:84 , srv702375681:84 , srv707429032:84 , srv741768988:84 , srv744697435:84 , srv757706899:84 , srv765933763:84 , srv776844843:84 , srv78386157:84 , srv805456621:84 , srv815555302:84 , srv821963354:84 , srv844191034:84 , srv849906287:84 , srv909751879:84 , srv922759971:84 , srv926624988:84 , srv931986728:84 , srv945162683:84 , srv947747894:84 , srv955010697:84 , srv95710794:84 , srv970538042:84 , srv973950304:84 , srv979000925:84 , srv1894423795:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-08 23:55:53,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:55:53,159 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1282308414=17, srv1116081893=9, srv614584620=74, srv1576918960=22, srv1359585843=19, srv1788413199=30, srv556682182=69, srv741768988=78, srv247039581=54, srv776844843=82, srv945162683=93, srv427209349=61, srv1551558945=21, srv1107132613=8, srv119809865=15, srv2076156349=48, srv757706899=80, srv1773119410=27, srv926624988=91, srv95710794=96, srv1784963943=29, srv1086825642=7, srv2133168291=52, srv1238463825=16, srv1959481559=44, srv1018151695=1, srv322700195=58, srv849906287=88, srv1990091384=45, srv1172362935=14, srv250278493=55, srv1075849561=5, srv955010697=95, srv1012442395=0, srv1999290689=46, srv1858398074=36, srv844191034=87, srv765933763=81, srv973950304=98, srv189391870=40, srv707429032=77, srv821963354=86, srv1794870019=31, srv257466230=56, srv1148897379=11, srv1362463723=20, srv430639069=62, srv1166178002=13, srv1725252724=24, srv78386157=83, srv105367128=3, srv473516507=64, srv805456621=84, srv931986728=92, srv591480086=73, srv909751879=89, srv1802903568=32, srv1154036226=12, srv1061561561=4, srv356567053=59, srv1750544602=26, srv465468185=63, srv1729094057=25, srv1922765778=42, srv1134965045=10, srv1946986734=43, srv493215069=67, srv979000925=99, srv407916686=60, srv2133478015=53, srv644390649=75, srv2132219180=51, srv1872036560=37, srv591369232=72, srv815555302=85, srv1587197620=23, srv482679894=66, srv922759971=90, srv1322964938=18, srv947747894=94, srv514400583=68, srv557114887=71, srv1873123471=38, srv1782023470=28, srv1827449449=34, srv1829083632=35, srv1894423795=41, srv2047975824=47, srv2086317914=49, srv1874961726=39, srv702375681=76, srv480171503=65, srv970538042=97, srv1079184834=6, srv316246632=57, srv1043954974=2, srv2097631809=50, srv744697435=79, srv182272299=33, srv556929091=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:55:53,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:55:53,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:55:53,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:55:53,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 23:55:53,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 23:55:53,263 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 23:55:53,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 23:55:53,265 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 23:55:53,266 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 23:55:53,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 23:55:53,268 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 23:55:53,549 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:53,549 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 23:55:53,550 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 23:55:54,188 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 897 ms to try 5000 different iterations.  Found a solution that moves 4692 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17844737913567. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04208199290016671, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05547266543577967, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5213333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04208199290016671, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23841666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:54,211 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv922759971:81 , srv1012442395:82 , srv1148897379:82 , srv1362463723:82 , srv182272299:82 , srv427209349:82 , srv947747894:82 , srv95710794:82 , srv2133168291:83 , srv407916686:83 , srv744697435:83 , srv1061561561:84 , srv1079184834:84 , srv1725252724:84 , srv1782023470:84 , srv1788413199:84 , srv1829083632:84 , srv1874961726:84 , srv1922765778:84 , srv1946986734:84 , srv247039581:84 , srv356567053:84 , srv557114887:84 , srv644390649:84 , srv78386157:84 , srv909751879:84 , srv931986728:84 , srv1043954974:85 , srv105367128:85 , srv1116081893:85 , srv1154036226:85 , srv1282308414:85 , srv1322964938:85 , srv1750544602:85 , srv1794870019:85 , srv1999290689:85 , srv2097631809:85 , srv316246632:85 , srv473516507:85 , srv493215069:85 , srv757706899:85 , srv765933763:85 , srv815555302:85 , srv821963354:85 , srv926624988:85 , srv973950304:85 , srv1107132613:86 , srv1134965045:86 , srv1551558945:86 , srv1802903568:86 , srv189391870:86 , srv322700195:86 , srv465468185:86 , srv741768988:86 , srv844191034:86 , srv979000925:86 , srv119809865:87 , srv1238463825:87 , srv1359585843:87 , srv1773119410:87 , srv1784963943:87 , srv1858398074:87 , srv1959481559:87 , srv2047975824:87 , srv2086317914:87 , srv2133478015:87 , srv480171503:87 , srv514400583:87 , srv556682182:87 , srv591369232:87 , srv955010697:87 , srv1086825642:88 , srv1576918960:88 , srv1872036560:88 , srv257466230:88 , srv614584620:88 , srv702375681:88 , srv776844843:88 , srv805456621:88 , srv970538042:88 , srv1018151695:89 , srv1166178002:89 , srv1587197620:89 , srv1827449449:89 , srv1873123471:89 , srv1990091384:89 , srv2076156349:89 , srv250278493:89 , srv430639069:89 , srv707429032:89 , srv1075849561:90 , srv1172362935:91 , srv2132219180:91 , srv482679894:91 , srv591480086:91 , srv849906287:91 , srv945162683:91 , srv1729094057:93 , srv556929091:97 , srv1894423795:464 }
2023-02-08 23:55:54,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:55:54,216 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1282308414=17, srv1116081893=9, srv614584620=74, srv1576918960=22, srv1359585843=19, srv1788413199=30, srv556682182=69, srv741768988=78, srv247039581=54, srv776844843=82, srv945162683=93, srv427209349=61, srv1551558945=21, srv1107132613=8, srv119809865=15, srv2076156349=48, srv757706899=80, srv1773119410=27, srv926624988=91, srv95710794=96, srv1784963943=29, srv1086825642=7, srv2133168291=52, srv1238463825=16, srv1959481559=44, srv1018151695=1, srv322700195=58, srv849906287=88, srv1990091384=45, srv1172362935=14, srv250278493=55, srv1075849561=5, srv955010697=95, srv1012442395=0, srv1999290689=46, srv1858398074=36, srv844191034=87, srv765933763=81, srv973950304=98, srv189391870=40, srv707429032=77, srv821963354=86, srv1794870019=31, srv257466230=56, srv1148897379=11, srv1362463723=20, srv430639069=62, srv1166178002=13, srv1725252724=24, srv78386157=83, srv105367128=3, srv473516507=64, srv805456621=84, srv931986728=92, srv591480086=73, srv909751879=89, srv1802903568=32, srv1154036226=12, srv1061561561=4, srv356567053=59, srv1750544602=26, srv465468185=63, srv1729094057=25, srv1922765778=42, srv1134965045=10, srv1946986734=43, srv493215069=67, srv979000925=99, srv407916686=60, srv2133478015=53, srv644390649=75, srv2132219180=51, srv1872036560=37, srv591369232=72, srv815555302=85, srv1587197620=23, srv482679894=66, srv922759971=90, srv1322964938=18, srv947747894=94, srv514400583=68, srv557114887=71, srv1873123471=38, srv1782023470=28, srv1827449449=34, srv1829083632=35, srv1894423795=41, srv2047975824=47, srv2086317914=49, srv1874961726=39, srv702375681=76, srv480171503=65, srv970538042=97, srv1079184834=6, srv316246632=57, srv1043954974=2, srv2097631809=50, srv744697435=79, srv182272299=33, srv556929091=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:55:54,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:55:54,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:55:54,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:55:54,234 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:55:54,235 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:55:54,236 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 23:55:54,237 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 23:55:54,238 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 23:55:54,239 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 23:55:54,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 23:55:54,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 23:55:54,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04208199290016671, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05547266543577967, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04208199290016671, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23841666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:54,454 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 23:55:54,454 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17841452504946553, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04208199290016671, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05547266543577967, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04208199290016671, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17383333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23841666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 23:55:54,772 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 510 ms to try 5000 different iterations.  Found a solution that moves 2582 regions; Going from a computed imbalance of 0.17841452504946553 to a new imbalance of 0.015206130189762766. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022060014052070618); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03714865336306437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2868888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022060014052070618); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043166666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:54,781 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1729094057:78 , srv945162683:78 , srv1873123471:81 , srv482679894:81 , srv556929091:81 , srv591480086:81 , srv707429032:81 , srv1018151695:82 , srv1075849561:82 , srv1086825642:82 , srv1172362935:82 , srv2132219180:82 , srv430639069:82 , srv849906287:82 , srv1784963943:83 , srv1990091384:83 , srv2076156349:83 , srv257466230:83 , srv514400583:83 , srv591369232:83 , srv614584620:83 , srv805456621:83 , srv1166178002:84 , srv1872036560:84 , srv1959481559:84 , srv250278493:84 , srv844191034:84 , srv1043954974:85 , srv1107132613:85 , srv1576918960:85 , srv1587197620:85 , srv1794870019:85 , srv1827449449:85 , srv2047975824:85 , srv2086317914:85 , srv322700195:85 , srv480171503:85 , srv702375681:85 , srv741768988:85 , srv776844843:85 , srv1079184834:86 , srv1116081893:86 , srv1134965045:86 , srv1154036226:86 , srv1238463825:86 , srv1282308414:86 , srv1359585843:86 , srv1551558945:86 , srv1782023470:86 , srv1829083632:86 , srv1858398074:86 , srv2133478015:86 , srv247039581:86 , srv465468185:86 , srv493215069:86 , srv556682182:86 , srv644390649:86 , srv765933763:86 , srv955010697:86 , srv973950304:86 , srv979000925:86 , srv1061561561:87 , srv1750544602:87 , srv1773119410:87 , srv1802903568:87 , srv1874961726:87 , srv1922765778:87 , srv2097631809:87 , srv815555302:87 , srv909751879:87 , srv931986728:87 , srv970538042:87 , srv105367128:88 , srv119809865:88 , srv1999290689:88 , srv316246632:88 , srv356567053:88 , srv473516507:88 , srv557114887:88 , srv744697435:88 , srv926624988:88 , srv1322964938:89 , srv1362463723:89 , srv1725252724:89 , srv1788413199:89 , srv182272299:89 , srv189391870:89 , srv2133168291:89 , srv427209349:89 , srv757706899:89 , srv1012442395:90 , srv407916686:90 , srv78386157:90 , srv821963354:90 , srv947747894:90 , srv95710794:90 , srv1148897379:91 , srv1946986734:91 , srv922759971:92 , srv1894423795:506 }
2023-02-08 23:55:54,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:55:54,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1282308414=17, srv1116081893=9, srv614584620=74, srv1576918960=22, srv1359585843=19, srv1788413199=30, srv556682182=69, srv741768988=78, srv247039581=54, srv776844843=82, srv945162683=93, srv427209349=61, srv1551558945=21, srv1107132613=8, srv119809865=15, srv2076156349=48, srv757706899=80, srv1773119410=27, srv926624988=91, srv95710794=96, srv1784963943=29, srv1086825642=7, srv2133168291=52, srv1238463825=16, srv1959481559=44, srv1018151695=1, srv322700195=58, srv849906287=88, srv1990091384=45, srv1172362935=14, srv250278493=55, srv1075849561=5, srv955010697=95, srv1012442395=0, srv1999290689=46, srv1858398074=36, srv844191034=87, srv765933763=81, srv973950304=98, srv189391870=40, srv707429032=77, srv821963354=86, srv1794870019=31, srv257466230=56, srv1148897379=11, srv1362463723=20, srv430639069=62, srv1166178002=13, srv1725252724=24, srv78386157=83, srv105367128=3, srv473516507=64, srv805456621=84, srv931986728=92, srv591480086=73, srv909751879=89, srv1802903568=32, srv1154036226=12, srv1061561561=4, srv356567053=59, srv1750544602=26, srv465468185=63, srv1729094057=25, srv1922765778=42, srv1134965045=10, srv1946986734=43, srv493215069=67, srv979000925=99, srv407916686=60, srv2133478015=53, srv644390649=75, srv2132219180=51, srv1872036560=37, srv591369232=72, srv815555302=85, srv1587197620=23, srv482679894=66, srv922759971=90, srv1322964938=18, srv947747894=94, srv514400583=68, srv557114887=71, srv1873123471=38, srv1782023470=28, srv1827449449=34, srv1829083632=35, srv1894423795=41, srv2047975824=47, srv2086317914=49, srv1874961726=39, srv702375681=76, srv480171503=65, srv970538042=97, srv1079184834=6, srv316246632=57, srv1043954974=2, srv2097631809=50, srv744697435=79, srv182272299=33, srv556929091=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:55:54,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:55:54,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 23:55:54,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 23:55:54,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 23:55:54,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 23:55:54,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 23:55:54,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 23:55:54,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 23:55:54,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022060014052070618); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03714865336306437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022060014052070618); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043166666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:54,930 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 23:55:54,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015188050639340787, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022060014052070618); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03714865336306437, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022060014052070618); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01225); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043166666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 23:55:55,149 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 321 ms to try 5000 different iterations.  Found a solution that moves 989 regions; Going from a computed imbalance of 0.015188050639340787 to a new imbalance of 7.180833344712457E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0064324746610942155); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007770559420976526); MoveCostFunction : (multiplier=7.0, imbalance=0.10988888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0064324746610942155); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:55,151 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv556929091:80 , srv1946986734:82 , srv821963354:82 , srv119809865:83 , srv1322964938:83 , srv1587197620:83 , srv1827449449:83 , srv757706899:83 , srv970538042:83 , srv1148897379:84 , srv1172362935:84 , srv1238463825:84 , srv1359585843:84 , srv1725252724:84 , srv1773119410:84 , srv1788413199:84 , srv189391870:84 , srv1999290689:84 , srv2132219180:84 , srv2133478015:84 , srv316246632:84 , srv407916686:84 , srv473516507:84 , srv702375681:84 , srv776844843:84 , srv78386157:84 , srv105367128:85 , srv1075849561:85 , srv1166178002:85 , srv1551558945:85 , srv1576918960:85 , srv1750544602:85 , srv1802903568:85 , srv1858398074:85 , srv1872036560:85 , srv2047975824:85 , srv2076156349:85 , srv2086317914:85 , srv250278493:85 , srv465468185:85 , srv480171503:85 , srv482679894:85 , srv493215069:85 , srv556682182:85 , srv557114887:85 , srv591480086:85 , srv815555302:85 , srv849906287:85 , srv922759971:85 , srv926624988:85 , srv955010697:85 , srv979000925:85 , srv1012442395:86 , srv1107132613:86 , srv1116081893:86 , srv1134965045:86 , srv1154036226:86 , srv1282308414:86 , srv1362463723:86 , srv182272299:86 , srv1922765778:86 , srv1990091384:86 , srv2097631809:86 , srv2133168291:86 , srv322700195:86 , srv427209349:86 , srv614584620:86 , srv741768988:86 , srv744697435:86 , srv805456621:86 , srv909751879:86 , srv95710794:86 , srv1018151695:87 , srv1043954974:87 , srv1061561561:87 , srv1086825642:87 , srv1729094057:87 , srv1782023470:87 , srv1784963943:87 , srv1794870019:87 , srv1829083632:87 , srv1873123471:87 , srv1874961726:87 , srv1959481559:87 , srv356567053:87 , srv430639069:87 , srv514400583:87 , srv644390649:87 , srv707429032:87 , srv765933763:87 , srv844191034:87 , srv947747894:87 , srv973950304:87 , srv1079184834:88 , srv247039581:88 , srv257466230:88 , srv591369232:88 , srv931986728:88 , srv945162683:88 , srv1894423795:545 }
2023-02-08 23:55:55,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:55:55,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1282308414=17, srv1116081893=9, srv614584620=74, srv1576918960=22, srv1359585843=19, srv1788413199=30, srv556682182=69, srv741768988=78, srv247039581=54, srv776844843=82, srv945162683=93, srv427209349=61, srv1551558945=21, srv1107132613=8, srv119809865=15, srv2076156349=48, srv757706899=80, srv1773119410=27, srv926624988=91, srv95710794=96, srv1784963943=29, srv1086825642=7, srv2133168291=52, srv1238463825=16, srv1959481559=44, srv1018151695=1, srv322700195=58, srv849906287=88, srv1990091384=45, srv1172362935=14, srv250278493=55, srv1075849561=5, srv955010697=95, srv1012442395=0, srv1999290689=46, srv1858398074=36, srv844191034=87, srv765933763=81, srv973950304=98, srv189391870=40, srv707429032=77, srv821963354=86, srv1794870019=31, srv257466230=56, srv1148897379=11, srv1362463723=20, srv430639069=62, srv1166178002=13, srv1725252724=24, srv78386157=83, srv105367128=3, srv473516507=64, srv805456621=84, srv931986728=92, srv591480086=73, srv909751879=89, srv1802903568=32, srv1154036226=12, srv1061561561=4, srv356567053=59, srv1750544602=26, srv465468185=63, srv1729094057=25, srv1922765778=42, srv1134965045=10, srv1946986734=43, srv493215069=67, srv979000925=99, srv407916686=60, srv2133478015=53, srv644390649=75, srv2132219180=51, srv1872036560=37, srv591369232=72, srv815555302=85, srv1587197620=23, srv482679894=66, srv922759971=90, srv1322964938=18, srv947747894=94, srv514400583=68, srv557114887=71, srv1873123471=38, srv1782023470=28, srv1827449449=34, srv1829083632=35, srv1894423795=41, srv2047975824=47, srv2086317914=49, srv1874961726=39, srv702375681=76, srv480171503=65, srv970538042=97, srv1079184834=6, srv316246632=57, srv1043954974=2, srv2097631809=50, srv744697435=79, srv182272299=33, srv556929091=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:55:55,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:55:55,167 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 23:55:55,168 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 23:55:55,169 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 23:55:55,170 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 23:55:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 23:55:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 23:55:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 23:55:55,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 23:55:55,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0064324746610942155); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007770559420976526); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0064324746610942155); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:55,274 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-08 23:55:55,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.111582084575595E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0064324746610942155); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007770559420976526); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0064324746610942155); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-08 23:55:55,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 414 ms to try 5000 different iterations.  Found a solution that moves 253 regions; Going from a computed imbalance of 7.111582084575595E-4 to a new imbalance of 2.0439461475855222E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.02811111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:55,595 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1018151695:83 , srv1729094057:83 , srv189391870:83 , srv257466230:83 , srv356567053:83 , srv947747894:83 , srv1012442395:84 , srv105367128:84 , srv1079184834:84 , srv1166178002:84 , srv119809865:84 , srv1576918960:84 , srv1773119410:84 , srv1794870019:84 , srv1802903568:84 , srv182272299:84 , srv1858398074:84 , srv1874961726:84 , srv1959481559:84 , srv1990091384:84 , srv2097631809:84 , srv2133168291:84 , srv2133478015:84 , srv247039581:84 , srv250278493:84 , srv316246632:84 , srv427209349:84 , srv430639069:84 , srv480171503:84 , srv514400583:84 , srv556682182:84 , srv591369232:84 , srv765933763:84 , srv78386157:84 , srv815555302:84 , srv849906287:84 , srv909751879:84 , srv922759971:84 , srv926624988:84 , srv931986728:84 , srv945162683:84 , srv955010697:84 , srv95710794:84 , srv970538042:84 , srv973950304:84 , srv1043954974:85 , srv1061561561:85 , srv1075849561:85 , srv1086825642:85 , srv1107132613:85 , srv1116081893:85 , srv1134965045:85 , srv1148897379:85 , srv1154036226:85 , srv1172362935:85 , srv1282308414:85 , srv1322964938:85 , srv1362463723:85 , srv1551558945:85 , srv1587197620:85 , srv1725252724:85 , srv1750544602:85 , srv1782023470:85 , srv1784963943:85 , srv1827449449:85 , srv1829083632:85 , srv1872036560:85 , srv1873123471:85 , srv1922765778:85 , srv1999290689:85 , srv2047975824:85 , srv2076156349:85 , srv2086317914:85 , srv2132219180:85 , srv322700195:85 , srv465468185:85 , srv473516507:85 , srv482679894:85 , srv556929091:85 , srv557114887:85 , srv591480086:85 , srv644390649:85 , srv702375681:85 , srv707429032:85 , srv741768988:85 , srv744697435:85 , srv757706899:85 , srv776844843:85 , srv821963354:85 , srv844191034:85 , srv979000925:85 , srv1238463825:86 , srv1359585843:86 , srv1788413199:86 , srv1946986734:86 , srv407916686:86 , srv493215069:86 , srv614584620:86 , srv805456621:86 , srv1894423795:628 }
2023-02-08 23:55:55,598 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:55:55,599 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1282308414=17, srv1116081893=9, srv614584620=74, srv1576918960=22, srv1359585843=19, srv1788413199=30, srv556682182=69, srv741768988=78, srv247039581=54, srv776844843=82, srv945162683=93, srv427209349=61, srv1551558945=21, srv1107132613=8, srv119809865=15, srv2076156349=48, srv757706899=80, srv1773119410=27, srv926624988=91, srv95710794=96, srv1784963943=29, srv1086825642=7, srv2133168291=52, srv1238463825=16, srv1959481559=44, srv1018151695=1, srv322700195=58, srv849906287=88, srv1990091384=45, srv1172362935=14, srv250278493=55, srv1075849561=5, srv955010697=95, srv1012442395=0, srv1999290689=46, srv1858398074=36, srv844191034=87, srv765933763=81, srv973950304=98, srv189391870=40, srv707429032=77, srv821963354=86, srv1794870019=31, srv257466230=56, srv1148897379=11, srv1362463723=20, srv430639069=62, srv1166178002=13, srv1725252724=24, srv78386157=83, srv105367128=3, srv473516507=64, srv805456621=84, srv931986728=92, srv591480086=73, srv909751879=89, srv1802903568=32, srv1154036226=12, srv1061561561=4, srv356567053=59, srv1750544602=26, srv465468185=63, srv1729094057=25, srv1922765778=42, srv1134965045=10, srv1946986734=43, srv493215069=67, srv979000925=99, srv407916686=60, srv2133478015=53, srv644390649=75, srv2132219180=51, srv1872036560=37, srv591369232=72, srv815555302=85, srv1587197620=23, srv482679894=66, srv922759971=90, srv1322964938=18, srv947747894=94, srv514400583=68, srv557114887=71, srv1873123471=38, srv1782023470=28, srv1827449449=34, srv1829083632=35, srv1894423795=41, srv2047975824=47, srv2086317914=49, srv1874961726=39, srv702375681=76, srv480171503=65, srv970538042=97, srv1079184834=6, srv316246632=57, srv1043954974=2, srv2097631809=50, srv744697435=79, srv182272299=33, srv556929091=70} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:55:55,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:55:55,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-08 23:55:55,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-08 23:55:55,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-08 23:55:55,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-08 23:55:55,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-08 23:55:55,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-08 23:55:55,716 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.866791761188899E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:55,717 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1018151695:83 , srv1729094057:83 , srv189391870:83 , srv257466230:83 , srv356567053:83 , srv947747894:83 , srv1012442395:84 , srv105367128:84 , srv1079184834:84 , srv1166178002:84 , srv119809865:84 , srv1576918960:84 , srv1773119410:84 , srv1794870019:84 , srv1802903568:84 , srv182272299:84 , srv1858398074:84 , srv1874961726:84 , srv1959481559:84 , srv1990091384:84 , srv2097631809:84 , srv2133168291:84 , srv2133478015:84 , srv247039581:84 , srv250278493:84 , srv316246632:84 , srv427209349:84 , srv430639069:84 , srv480171503:84 , srv514400583:84 , srv556682182:84 , srv591369232:84 , srv765933763:84 , srv78386157:84 , srv815555302:84 , srv849906287:84 , srv909751879:84 , srv922759971:84 , srv926624988:84 , srv931986728:84 , srv945162683:84 , srv955010697:84 , srv95710794:84 , srv970538042:84 , srv973950304:84 , srv1043954974:85 , srv1061561561:85 , srv1075849561:85 , srv1086825642:85 , srv1107132613:85 , srv1116081893:85 , srv1134965045:85 , srv1148897379:85 , srv1154036226:85 , srv1172362935:85 , srv1282308414:85 , srv1322964938:85 , srv1362463723:85 , srv1551558945:85 , srv1587197620:85 , srv1725252724:85 , srv1750544602:85 , srv1782023470:85 , srv1784963943:85 , srv1827449449:85 , srv1829083632:85 , srv1872036560:85 , srv1873123471:85 , srv1922765778:85 , srv1999290689:85 , srv2047975824:85 , srv2076156349:85 , srv2086317914:85 , srv2132219180:85 , srv322700195:85 , srv465468185:85 , srv473516507:85 , srv482679894:85 , srv556929091:85 , srv557114887:85 , srv591480086:85 , srv644390649:85 , srv702375681:85 , srv707429032:85 , srv741768988:85 , srv744697435:85 , srv757706899:85 , srv776844843:85 , srv821963354:85 , srv844191034:85 , srv979000925:85 , srv1238463825:86 , srv1359585843:86 , srv1788413199:86 , srv1946986734:86 , srv407916686:86 , srv493215069:86 , srv614584620:86 , srv805456621:86 , srv1894423795:628 }
2023-02-08 23:55:55,742 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=100 (was 100), ProcessCount=168 (was 168), AvailableMemoryMB=3393 (was 3878)
2023-02-08 23:55:55,749 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=168, AvailableMemoryMB=3393
2023-02-08 23:55:55,749 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-08 23:55:55,750 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-08 23:55:55,763 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1180616216:84 , srv1301245939:84 , srv1349389214:84 , srv1361803728:84 , srv1583514077:84 , srv1644538980:84 , srv1689319046:84 , srv181898072:84 , srv1862145516:84 , srv1894227917:84 , srv1910943517:84 , srv1912882017:84 , srv2117931562:84 , srv2127976159:84 , srv2131044966:84 , srv2133191608:84 , srv282523622:84 , srv310950833:84 , srv36694141:84 , srv413970490:84 , srv440853021:84 , srv553720969:84 , srv560821245:84 , srv592106937:84 , srv593448451:84 , srv64886485:84 , srv681554830:84 , srv727294994:84 , srv957469489:84 , srv6914524:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-08 23:55:55,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:55:55,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1301245939=1, srv957469489=29, srv1180616216=0, srv2131044966=14, srv413970490=19, srv1644538980=5, srv1910943517=10, srv282523622=16, srv593448451=24, srv64886485=25, srv2133191608=15, srv560821245=22, srv2127976159=13, srv310950833=17, srv1361803728=3, srv2117931562=12, srv6914524=27, srv1912882017=11, srv36694141=18, srv181898072=7, srv592106937=23, srv681554830=26, srv1689319046=6, srv440853021=20, srv1862145516=8, srv553720969=21, srv1349389214=2, srv1583514077=4, srv1894227917=9, srv727294994=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:55:55,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:55:55,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-08 23:55:55,792 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2089565594490024, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:55:55,793 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912851458560042, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.2089565594490024, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-08 23:57:55,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6510765 different iterations.  Found a solution that moves 1857 regions; Going from a computed imbalance of 0.9912851458560042 to a new imbalance of 4.334330639506329E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6877777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:57:55,785 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1180616216:90 , srv1301245939:90 , srv1349389214:90 , srv1361803728:90 , srv1583514077:90 , srv1644538980:90 , srv1689319046:90 , srv181898072:90 , srv1862145516:90 , srv1894227917:90 , srv1910943517:90 , srv1912882017:90 , srv2117931562:90 , srv2127976159:90 , srv2131044966:90 , srv2133191608:90 , srv282523622:90 , srv310950833:90 , srv36694141:90 , srv413970490:90 , srv440853021:90 , srv553720969:90 , srv560821245:90 , srv592106937:90 , srv593448451:90 , srv64886485:90 , srv681554830:90 , srv6914524:90 , srv727294994:90 , srv957469489:90 }
2023-02-08 23:57:55,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-08 23:57:55,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1301245939=1, srv957469489=29, srv1180616216=0, srv2131044966=14, srv413970490=19, srv1644538980=5, srv1910943517=10, srv282523622=16, srv593448451=24, srv64886485=25, srv2133191608=15, srv560821245=22, srv2127976159=13, srv310950833=17, srv1361803728=3, srv2117931562=12, srv6914524=27, srv1912882017=11, srv36694141=18, srv181898072=7, srv592106937=23, srv681554830=26, srv1689319046=6, srv440853021=20, srv1862145516=8, srv553720969=21, srv1349389214=2, srv1583514077=4, srv1894227917=9, srv727294994=28} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-08 23:57:55,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-08 23:57:55,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-08 23:57:55,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-08 23:57:55,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-08 23:57:55,812 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-08 23:57:55,812 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1180616216:90 , srv1301245939:90 , srv1349389214:90 , srv1361803728:90 , srv1583514077:90 , srv1644538980:90 , srv1689319046:90 , srv181898072:90 , srv1862145516:90 , srv1894227917:90 , srv1910943517:90 , srv1912882017:90 , srv2117931562:90 , srv2127976159:90 , srv2131044966:90 , srv2133191608:90 , srv282523622:90 , srv310950833:90 , srv36694141:90 , srv413970490:90 , srv440853021:90 , srv553720969:90 , srv560821245:90 , srv592106937:90 , srv593448451:90 , srv64886485:90 , srv681554830:90 , srv6914524:90 , srv727294994:90 , srv957469489:90 }
2023-02-08 23:57:55,827 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=202 (was 100) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3349 (was 3393)