2024-01-07 10:52:49,105 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-01-07 10:52:49,387 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-01-07 10:52:49,493 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-01-07 10:52:49,554 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-07 10:52:49,561 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-07 10:52:49,574 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=438, ProcessCount=174, AvailableMemoryMB=7528
2024-01-07 10:52:49,575 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-07 10:52:49,576 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-01-07 10:52:49,720 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015004694:84 , srv1039959680:84 , srv1079262860:84 , srv1097109054:84 , srv1120224247:84 , srv1122395035:84 , srv1130047613:84 , srv1149239791:84 , srv1154506474:84 , srv1184396574:84 , srv1187379230:84 , srv1203989242:84 , srv1222642441:84 , srv1226953303:84 , srv1239098064:84 , srv1239373370:84 , srv1286032871:84 , srv1289099111:84 , srv1293365565:84 , srv1311387210:84 , srv1325646896:84 , srv1398772473:84 , srv1403363011:84 , srv1418655453:84 , srv1425541511:84 , srv1438827097:84 , srv1477421584:84 , srv1492228624:84 , srv1498480289:84 , srv1541850672:84 , srv1556795381:84 , srv1618933639:84 , srv1640709060:84 , srv1642721647:84 , srv166700373:84 , srv1676797220:84 , srv1677768500:84 , srv1693080769:84 , srv1697595955:84 , srv1710676532:84 , srv1718218976:84 , srv1759412053:84 , srv1773610557:84 , srv17784065:84 , srv1809960439:84 , srv1818168062:84 , srv1829103831:84 , srv1862210839:84 , srv1881763717:84 , srv1941237508:84 , srv1998948522:84 , srv2006081050:84 , srv2041411728:84 , srv2076399713:84 , srv2142112483:84 , srv2144597750:84 , srv227760643:84 , srv258469760:84 , srv260117115:84 , srv261210313:84 , srv265806244:84 , srv339698696:84 , srv347226327:84 , srv401316066:84 , srv402176303:84 , srv427336737:84 , srv472689019:84 , srv482837601:84 , srv490954144:84 , srv492583965:84 , srv524566156:84 , srv525593896:84 , srv550899740:84 , srv592093075:84 , srv625839411:84 , srv626057155:84 , srv631993028:84 , srv655368522:84 , srv655504846:84 , srv65981265:84 , srv68140221:84 , srv696301785:84 , srv707637768:84 , srv711015885:84 , srv729791928:84 , srv742439258:84 , srv746577497:84 , srv755558156:84 , srv801515188:84 , srv802559383:84 , srv821306737:84 , srv838303189:84 , srv840613855:84 , srv857701243:84 , srv868217043:84 , srv893441323:84 , srv897645221:84 , srv91502798:84 , srv970915130:84 , srv867233101:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-01-07 10:52:49,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:49,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv260117115=58, srv1418655453=23, srv1541850672=29, srv1120224247=4, srv1818168062=45, srv91502798=98, srv592093075=73, srv625839411=74, srv742439258=85, srv402176303=64, srv2144597750=55, srv1718218976=40, srv1477421584=26, srv970915130=99, srv492583965=69, srv65981265=79, srv427336737=65, srv1289099111=17, srv1640709060=32, srv482837601=67, srv1881763717=48, srv1492228624=27, srv1079262860=2, srv261210313=59, srv401316066=63, srv897645221=97, srv490954144=68, srv707637768=82, srv1015004694=0, srv893441323=96, srv227760643=56, srv1398772473=21, srv339698696=61, srv1184396574=9, srv868217043=95, srv1130047613=6, srv1203989242=11, srv1438827097=25, srv755558156=87, srv1222642441=12, srv550899740=72, srv1149239791=7, srv1642721647=33, srv867233101=94, srv801515188=88, srv746577497=86, srv1239098064=14, srv166700373=34, srv1710676532=39, srv524566156=70, srv1039959680=1, srv626057155=75, srv655504846=78, srv1122395035=5, srv1829103831=46, srv1773610557=42, srv2006081050=51, srv1286032871=16, srv1809960439=44, srv696301785=81, srv265806244=60, srv1425541511=24, srv1325646896=20, srv821306737=90, srv17784065=43, srv711015885=83, srv1097109054=3, srv1403363011=22, srv1226953303=13, srv1862210839=47, srv1998948522=50, srv2142112483=54, srv1697595955=38, srv802559383=89, srv1239373370=15, srv655368522=77, srv1759412053=41, srv840613855=92, srv1556795381=30, srv1187379230=10, srv631993028=76, srv1293365565=18, srv1676797220=35, srv857701243=93, srv347226327=62, srv1618933639=31, srv472689019=66, srv1311387210=19, srv838303189=91, srv729791928=84, srv68140221=80, srv1941237508=49, srv1154506474=8, srv2041411728=52, srv525593896=71, srv1677768500=36, srv1693080769=37, srv1498480289=28, srv2076399713=53, srv258469760=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:49,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:49,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:49,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:49,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:49,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-07 10:52:49,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-07 10:52:49,805 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-07 10:52:49,806 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-07 10:52:49,807 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-07 10:52:49,808 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-07 10:52:49,809 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-07 10:52:49,810 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-07 10:52:49,963 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:49,963 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-07 10:52:49,964 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-07 10:52:50,271 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 441 ms to try 5000 different iterations.  Found a solution that moves 4694 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1753152534910743. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038269070550747075, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05540180745571636, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5215555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038269070550747075, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17083333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23383333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:50,295 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1286032871:83 , srv1492228624:83 , srv1710676532:83 , srv1718218976:83 , srv227760643:83 , srv339698696:83 , srv427336737:83 , srv625839411:83 , srv626057155:83 , srv1149239791:84 , srv1154506474:84 , srv1289099111:84 , srv1325646896:84 , srv1498480289:84 , srv1618933639:84 , srv1676797220:84 , srv1809960439:84 , srv1862210839:84 , srv261210313:84 , srv402176303:84 , srv65981265:84 , srv68140221:84 , srv801515188:84 , srv857701243:84 , srv893441323:84 , srv1015004694:85 , srv1187379230:85 , srv1203989242:85 , srv1239098064:85 , srv1311387210:85 , srv1693080769:85 , srv1697595955:85 , srv1759412053:85 , srv1941237508:85 , srv2006081050:85 , srv2076399713:85 , srv265806244:85 , srv482837601:85 , srv550899740:85 , srv592093075:85 , srv655504846:85 , srv838303189:85 , srv897645221:85 , srv1097109054:86 , srv1122395035:86 , srv1398772473:86 , srv1477421584:86 , srv1642721647:86 , srv1677768500:86 , srv1818168062:86 , srv1829103831:86 , srv2041411728:86 , srv472689019:86 , srv524566156:86 , srv696301785:86 , srv91502798:86 , srv970915130:86 , srv1226953303:87 , srv1425541511:87 , srv166700373:87 , srv2144597750:87 , srv401316066:87 , srv525593896:87 , srv655368522:87 , srv707637768:87 , srv729791928:87 , srv1039959680:88 , srv258469760:88 , srv492583965:88 , srv746577497:88 , srv802559383:88 , srv1120224247:89 , srv1130047613:89 , srv1184396574:89 , srv1222642441:89 , srv1773610557:89 , srv17784065:89 , srv260117115:89 , srv347226327:89 , srv755558156:89 , srv868217043:89 , srv1239373370:90 , srv1293365565:90 , srv1403363011:90 , srv1438827097:90 , srv1640709060:90 , srv1881763717:90 , srv1998948522:90 , srv2142112483:90 , srv490954144:90 , srv631993028:90 , srv711015885:90 , srv821306737:90 , srv840613855:90 , srv1418655453:91 , srv1541850672:92 , srv1556795381:92 , srv742439258:92 , srv1079262860:95 , srv867233101:430 }
2024-01-07 10:52:50,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:50,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv260117115=58, srv1418655453=23, srv1541850672=29, srv1120224247=4, srv1818168062=45, srv91502798=98, srv592093075=73, srv625839411=74, srv742439258=85, srv402176303=64, srv2144597750=55, srv1718218976=40, srv1477421584=26, srv970915130=99, srv492583965=69, srv65981265=79, srv427336737=65, srv1289099111=17, srv1640709060=32, srv482837601=67, srv1881763717=48, srv1492228624=27, srv1079262860=2, srv261210313=59, srv401316066=63, srv897645221=97, srv490954144=68, srv707637768=82, srv1015004694=0, srv893441323=96, srv227760643=56, srv1398772473=21, srv339698696=61, srv1184396574=9, srv868217043=95, srv1130047613=6, srv1203989242=11, srv1438827097=25, srv755558156=87, srv1222642441=12, srv550899740=72, srv1149239791=7, srv1642721647=33, srv867233101=94, srv801515188=88, srv746577497=86, srv1239098064=14, srv166700373=34, srv1710676532=39, srv524566156=70, srv1039959680=1, srv626057155=75, srv655504846=78, srv1122395035=5, srv1829103831=46, srv1773610557=42, srv2006081050=51, srv1286032871=16, srv1809960439=44, srv696301785=81, srv265806244=60, srv1425541511=24, srv1325646896=20, srv821306737=90, srv17784065=43, srv711015885=83, srv1097109054=3, srv1403363011=22, srv1226953303=13, srv1862210839=47, srv1998948522=50, srv2142112483=54, srv1697595955=38, srv802559383=89, srv1239373370=15, srv655368522=77, srv1759412053=41, srv840613855=92, srv1556795381=30, srv1187379230=10, srv631993028=76, srv1293365565=18, srv1676797220=35, srv857701243=93, srv347226327=62, srv1618933639=31, srv472689019=66, srv1311387210=19, srv838303189=91, srv729791928=84, srv68140221=80, srv1941237508=49, srv1154506474=8, srv2041411728=52, srv525593896=71, srv1677768500=36, srv1693080769=37, srv1498480289=28, srv2076399713=53, srv258469760=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:50,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:50,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:50,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:50,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:50,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:50,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:50,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:50,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:50,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:50,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:50,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-07 10:52:50,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-07 10:52:50,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-07 10:52:50,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-07 10:52:50,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-07 10:52:50,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-07 10:52:50,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-07 10:52:50,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-07 10:52:50,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-07 10:52:50,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-07 10:52:50,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-07 10:52:50,503 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038269070550747075, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05540180745571636, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038269070550747075, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17083333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23383333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:50,504 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-07 10:52:50,504 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17528238540057053, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.038269070550747075, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05540180745571636, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.038269070550747075, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17083333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23383333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-07 10:52:50,705 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 335 ms to try 5000 different iterations.  Found a solution that moves 2681 regions; Going from a computed imbalance of 0.17528238540057053 to a new imbalance of 0.014831829178591464. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021949505021062764); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03396043119049315, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29788888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021949505021062764); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:50,717 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1079262860:76 , srv742439258:77 , srv1556795381:78 , srv1541850672:80 , srv1998948522:80 , srv711015885:80 , srv1039959680:81 , srv1184396574:81 , srv1222642441:81 , srv1239373370:81 , srv1403363011:81 , srv1293365565:82 , srv1425541511:82 , srv1640709060:82 , srv1881763717:82 , srv2142112483:82 , srv631993028:82 , srv821306737:82 , srv840613855:82 , srv1418655453:83 , srv258469760:83 , srv492583965:83 , srv755558156:83 , srv1120224247:84 , srv1226953303:84 , srv1398772473:84 , srv1773610557:84 , srv260117115:84 , srv265806244:84 , srv472689019:84 , srv490954144:84 , srv524566156:84 , srv746577497:84 , srv868217043:84 , srv1097109054:85 , srv1130047613:85 , srv1187379230:85 , srv1438827097:85 , srv166700373:85 , srv1677768500:85 , srv1697595955:85 , srv1759412053:85 , srv1818168062:85 , srv2006081050:85 , srv2041411728:85 , srv347226327:85 , srv482837601:85 , srv655368522:85 , srv729791928:85 , srv1203989242:86 , srv1477421584:86 , srv2076399713:86 , srv2144597750:86 , srv401316066:86 , srv402176303:86 , srv525593896:86 , srv707637768:86 , srv970915130:86 , srv1015004694:87 , srv1122395035:87 , srv1154506474:87 , srv1642721647:87 , srv1693080769:87 , srv17784065:87 , srv1829103831:87 , srv1941237508:87 , srv427336737:87 , srv655504846:87 , srv696301785:87 , srv802559383:87 , srv91502798:87 , srv1311387210:88 , srv1325646896:88 , srv1618933639:88 , srv1862210839:88 , srv261210313:88 , srv592093075:88 , srv65981265:88 , srv801515188:88 , srv838303189:88 , srv1149239791:89 , srv1289099111:89 , srv1492228624:89 , srv1498480289:89 , srv1676797220:89 , srv1710676532:89 , srv1718218976:89 , srv339698696:89 , srv550899740:89 , srv625839411:89 , srv1239098064:90 , srv1286032871:90 , srv1809960439:90 , srv227760643:90 , srv626057155:90 , srv857701243:90 , srv893441323:90 , srv897645221:90 , srv68140221:91 , srv867233101:539 }
2024-01-07 10:52:50,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:50,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv260117115=58, srv1418655453=23, srv1541850672=29, srv1120224247=4, srv1818168062=45, srv91502798=98, srv592093075=73, srv625839411=74, srv742439258=85, srv402176303=64, srv2144597750=55, srv1718218976=40, srv1477421584=26, srv970915130=99, srv492583965=69, srv65981265=79, srv427336737=65, srv1289099111=17, srv1640709060=32, srv482837601=67, srv1881763717=48, srv1492228624=27, srv1079262860=2, srv261210313=59, srv401316066=63, srv897645221=97, srv490954144=68, srv707637768=82, srv1015004694=0, srv893441323=96, srv227760643=56, srv1398772473=21, srv339698696=61, srv1184396574=9, srv868217043=95, srv1130047613=6, srv1203989242=11, srv1438827097=25, srv755558156=87, srv1222642441=12, srv550899740=72, srv1149239791=7, srv1642721647=33, srv867233101=94, srv801515188=88, srv746577497=86, srv1239098064=14, srv166700373=34, srv1710676532=39, srv524566156=70, srv1039959680=1, srv626057155=75, srv655504846=78, srv1122395035=5, srv1829103831=46, srv1773610557=42, srv2006081050=51, srv1286032871=16, srv1809960439=44, srv696301785=81, srv265806244=60, srv1425541511=24, srv1325646896=20, srv821306737=90, srv17784065=43, srv711015885=83, srv1097109054=3, srv1403363011=22, srv1226953303=13, srv1862210839=47, srv1998948522=50, srv2142112483=54, srv1697595955=38, srv802559383=89, srv1239373370=15, srv655368522=77, srv1759412053=41, srv840613855=92, srv1556795381=30, srv1187379230=10, srv631993028=76, srv1293365565=18, srv1676797220=35, srv857701243=93, srv347226327=62, srv1618933639=31, srv472689019=66, srv1311387210=19, srv838303189=91, srv729791928=84, srv68140221=80, srv1941237508=49, srv1154506474=8, srv2041411728=52, srv525593896=71, srv1677768500=36, srv1693080769=37, srv1498480289=28, srv2076399713=53, srv258469760=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:50,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:50,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:50,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:50,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:50,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:50,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-07 10:52:50,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-07 10:52:50,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-07 10:52:50,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-07 10:52:50,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-07 10:52:50,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-07 10:52:50,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-07 10:52:50,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-07 10:52:50,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-07 10:52:50,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021949505021062764); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03396043119049315, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021949505021062764); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:50,850 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-07 10:52:50,850 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.01481305641535315, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021949505021062764); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03396043119049315, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021949505021062764); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04416666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-07 10:52:51,008 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 235 ms to try 5000 different iterations.  Found a solution that moves 1009 regions; Going from a computed imbalance of 0.01481305641535315 to a new imbalance of 5.302517947054859E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.003985883558148631); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006963106238227913); MoveCostFunction : (multiplier=7.0, imbalance=0.1121111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.003985883558148631); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,010 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1438827097:82 , srv17784065:82 , srv1239098064:83 , srv68140221:83 , srv802559383:83 , srv897645221:83 , srv1122395035:84 , srv1130047613:84 , srv1289099111:84 , srv1418655453:84 , srv1642721647:84 , srv1773610557:84 , srv1809960439:84 , srv525593896:84 , srv550899740:84 , srv857701243:84 , srv893441323:84 , srv91502798:84 , srv1120224247:85 , srv1149239791:85 , srv1286032871:85 , srv1311387210:85 , srv1477421584:85 , srv1498480289:85 , srv1640709060:85 , srv1676797220:85 , srv1718218976:85 , srv1829103831:85 , srv1881763717:85 , srv2142112483:85 , srv2144597750:85 , srv227760643:85 , srv261210313:85 , srv339698696:85 , srv347226327:85 , srv401316066:85 , srv490954144:85 , srv625839411:85 , srv626057155:85 , srv65981265:85 , srv696301785:85 , srv707637768:85 , srv729791928:85 , srv821306737:85 , srv838303189:85 , srv840613855:85 , srv868217043:85 , srv1015004694:86 , srv1079262860:86 , srv1097109054:86 , srv1154506474:86 , srv1226953303:86 , srv1239373370:86 , srv1293365565:86 , srv1325646896:86 , srv1403363011:86 , srv1492228624:86 , srv1618933639:86 , srv166700373:86 , srv1693080769:86 , srv1710676532:86 , srv1818168062:86 , srv1862210839:86 , srv2076399713:86 , srv258469760:86 , srv260117115:86 , srv492583965:86 , srv592093075:86 , srv631993028:86 , srv655368522:86 , srv655504846:86 , srv746577497:86 , srv755558156:86 , srv801515188:86 , srv970915130:86 , srv1184396574:87 , srv1203989242:87 , srv1398772473:87 , srv1541850672:87 , srv1556795381:87 , srv1677768500:87 , srv1759412053:87 , srv1941237508:87 , srv1998948522:87 , srv2006081050:87 , srv402176303:87 , srv427336737:87 , srv472689019:87 , srv524566156:87 , srv1039959680:88 , srv1187379230:88 , srv1222642441:88 , srv1425541511:88 , srv1697595955:88 , srv2041411728:88 , srv482837601:88 , srv711015885:88 , srv742439258:88 , srv265806244:89 , srv867233101:524 }
2024-01-07 10:52:51,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:51,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv260117115=58, srv1418655453=23, srv1541850672=29, srv1120224247=4, srv1818168062=45, srv91502798=98, srv592093075=73, srv625839411=74, srv742439258=85, srv402176303=64, srv2144597750=55, srv1718218976=40, srv1477421584=26, srv970915130=99, srv492583965=69, srv65981265=79, srv427336737=65, srv1289099111=17, srv1640709060=32, srv482837601=67, srv1881763717=48, srv1492228624=27, srv1079262860=2, srv261210313=59, srv401316066=63, srv897645221=97, srv490954144=68, srv707637768=82, srv1015004694=0, srv893441323=96, srv227760643=56, srv1398772473=21, srv339698696=61, srv1184396574=9, srv868217043=95, srv1130047613=6, srv1203989242=11, srv1438827097=25, srv755558156=87, srv1222642441=12, srv550899740=72, srv1149239791=7, srv1642721647=33, srv867233101=94, srv801515188=88, srv746577497=86, srv1239098064=14, srv166700373=34, srv1710676532=39, srv524566156=70, srv1039959680=1, srv626057155=75, srv655504846=78, srv1122395035=5, srv1829103831=46, srv1773610557=42, srv2006081050=51, srv1286032871=16, srv1809960439=44, srv696301785=81, srv265806244=60, srv1425541511=24, srv1325646896=20, srv821306737=90, srv17784065=43, srv711015885=83, srv1097109054=3, srv1403363011=22, srv1226953303=13, srv1862210839=47, srv1998948522=50, srv2142112483=54, srv1697595955=38, srv802559383=89, srv1239373370=15, srv655368522=77, srv1759412053=41, srv840613855=92, srv1556795381=30, srv1187379230=10, srv631993028=76, srv1293365565=18, srv1676797220=35, srv857701243=93, srv347226327=62, srv1618933639=31, srv472689019=66, srv1311387210=19, srv838303189=91, srv729791928=84, srv68140221=80, srv1941237508=49, srv1154506474=8, srv2041411728=52, srv525593896=71, srv1677768500=36, srv1693080769=37, srv1498480289=28, srv2076399713=53, srv258469760=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:51,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:51,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-07 10:52:51,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-07 10:52:51,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-07 10:52:51,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-07 10:52:51,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-07 10:52:51,031 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-07 10:52:51,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-07 10:52:51,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-07 10:52:51,121 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.003985883558148631); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006963106238227913); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.003985883558148631); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,121 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-07 10:52:51,121 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.231866256986007E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.003985883558148631); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006963106238227913); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.003985883558148631); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-07 10:52:51,361 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 321 ms to try 5000 different iterations.  Found a solution that moves 268 regions; Going from a computed imbalance of 5.231866256986007E-4 to a new imbalance of 4.346379482286149E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.029777777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,362 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv347226327:82 , srv1130047613:83 , srv1693080769:83 , srv1941237508:83 , srv2041411728:83 , srv260117115:83 , srv490954144:83 , srv592093075:83 , srv655504846:83 , srv1015004694:84 , srv1149239791:84 , srv1187379230:84 , srv1203989242:84 , srv1222642441:84 , srv1239098064:84 , srv1286032871:84 , srv1293365565:84 , srv1311387210:84 , srv1325646896:84 , srv1403363011:84 , srv1418655453:84 , srv1477421584:84 , srv1492228624:84 , srv1541850672:84 , srv166700373:84 , srv1676797220:84 , srv1677768500:84 , srv1697595955:84 , srv1710676532:84 , srv17784065:84 , srv1809960439:84 , srv1862210839:84 , srv1881763717:84 , srv1998948522:84 , srv2144597750:84 , srv227760643:84 , srv265806244:84 , srv401316066:84 , srv482837601:84 , srv550899740:84 , srv625839411:84 , srv626057155:84 , srv631993028:84 , srv655368522:84 , srv68140221:84 , srv696301785:84 , srv707637768:84 , srv711015885:84 , srv746577497:84 , srv755558156:84 , srv802559383:84 , srv838303189:84 , srv868217043:84 , srv893441323:84 , srv897645221:84 , srv970915130:84 , srv1039959680:85 , srv1079262860:85 , srv1097109054:85 , srv1120224247:85 , srv1122395035:85 , srv1154506474:85 , srv1184396574:85 , srv1226953303:85 , srv1239373370:85 , srv1289099111:85 , srv1398772473:85 , srv1425541511:85 , srv1438827097:85 , srv1498480289:85 , srv1556795381:85 , srv1618933639:85 , srv1642721647:85 , srv1718218976:85 , srv1759412053:85 , srv1773610557:85 , srv1818168062:85 , srv1829103831:85 , srv2006081050:85 , srv2076399713:85 , srv2142112483:85 , srv258469760:85 , srv261210313:85 , srv339698696:85 , srv402176303:85 , srv427336737:85 , srv472689019:85 , srv492583965:85 , srv524566156:85 , srv525593896:85 , srv65981265:85 , srv729791928:85 , srv801515188:85 , srv821306737:85 , srv840613855:85 , srv857701243:85 , srv91502798:85 , srv1640709060:86 , srv742439258:86 , srv867233101:649 }
2024-01-07 10:52:51,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:51,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv260117115=58, srv1418655453=23, srv1541850672=29, srv1120224247=4, srv1818168062=45, srv91502798=98, srv592093075=73, srv625839411=74, srv742439258=85, srv402176303=64, srv2144597750=55, srv1718218976=40, srv1477421584=26, srv970915130=99, srv492583965=69, srv65981265=79, srv427336737=65, srv1289099111=17, srv1640709060=32, srv482837601=67, srv1881763717=48, srv1492228624=27, srv1079262860=2, srv261210313=59, srv401316066=63, srv897645221=97, srv490954144=68, srv707637768=82, srv1015004694=0, srv893441323=96, srv227760643=56, srv1398772473=21, srv339698696=61, srv1184396574=9, srv868217043=95, srv1130047613=6, srv1203989242=11, srv1438827097=25, srv755558156=87, srv1222642441=12, srv550899740=72, srv1149239791=7, srv1642721647=33, srv867233101=94, srv801515188=88, srv746577497=86, srv1239098064=14, srv166700373=34, srv1710676532=39, srv524566156=70, srv1039959680=1, srv626057155=75, srv655504846=78, srv1122395035=5, srv1829103831=46, srv1773610557=42, srv2006081050=51, srv1286032871=16, srv1809960439=44, srv696301785=81, srv265806244=60, srv1425541511=24, srv1325646896=20, srv821306737=90, srv17784065=43, srv711015885=83, srv1097109054=3, srv1403363011=22, srv1226953303=13, srv1862210839=47, srv1998948522=50, srv2142112483=54, srv1697595955=38, srv802559383=89, srv1239373370=15, srv655368522=77, srv1759412053=41, srv840613855=92, srv1556795381=30, srv1187379230=10, srv631993028=76, srv1293365565=18, srv1676797220=35, srv857701243=93, srv347226327=62, srv1618933639=31, srv472689019=66, srv1311387210=19, srv838303189=91, srv729791928=84, srv68140221=80, srv1941237508=49, srv1154506474=8, srv2041411728=52, srv525593896=71, srv1677768500=36, srv1693080769=37, srv1498480289=28, srv2076399713=53, srv258469760=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:51,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:51,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-07 10:52:51,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-07 10:52:51,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-07 10:52:51,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-07 10:52:51,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-07 10:52:51,463 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,464 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-01-07 10:52:51,464 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.158721871399608E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.737793696791342E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.737793696791342E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-01-07 10:52:51,609 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 222 ms to try 5000 different iterations.  Found a solution that moves 39 regions; Going from a computed imbalance of 4.158721871399608E-5 to a new imbalance of 6.671069509007577E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.004333333333333333); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,609 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1120224247:83 , srv1498480289:83 , srv1541850672:83 , srv1618933639:83 , srv1640709060:83 , srv1829103831:83 , srv742439258:83 , srv801515188:83 , srv857701243:83 , srv1015004694:84 , srv1039959680:84 , srv1079262860:84 , srv1097109054:84 , srv1122395035:84 , srv1149239791:84 , srv1154506474:84 , srv1184396574:84 , srv1187379230:84 , srv1203989242:84 , srv1222642441:84 , srv1226953303:84 , srv1239098064:84 , srv1239373370:84 , srv1286032871:84 , srv1289099111:84 , srv1293365565:84 , srv1311387210:84 , srv1325646896:84 , srv1398772473:84 , srv1418655453:84 , srv1425541511:84 , srv1438827097:84 , srv1492228624:84 , srv1556795381:84 , srv1642721647:84 , srv166700373:84 , srv1676797220:84 , srv1677768500:84 , srv1697595955:84 , srv1710676532:84 , srv1718218976:84 , srv1759412053:84 , srv1773610557:84 , srv17784065:84 , srv1809960439:84 , srv1818168062:84 , srv1862210839:84 , srv1941237508:84 , srv2006081050:84 , srv2041411728:84 , srv2076399713:84 , srv2142112483:84 , srv2144597750:84 , srv227760643:84 , srv258469760:84 , srv260117115:84 , srv261210313:84 , srv265806244:84 , srv339698696:84 , srv401316066:84 , srv402176303:84 , srv427336737:84 , srv472689019:84 , srv482837601:84 , srv490954144:84 , srv492583965:84 , srv524566156:84 , srv525593896:84 , srv550899740:84 , srv592093075:84 , srv626057155:84 , srv631993028:84 , srv655368522:84 , srv65981265:84 , srv68140221:84 , srv696301785:84 , srv707637768:84 , srv711015885:84 , srv729791928:84 , srv746577497:84 , srv755558156:84 , srv802559383:84 , srv821306737:84 , srv838303189:84 , srv840613855:84 , srv868217043:84 , srv893441323:84 , srv897645221:84 , srv91502798:84 , srv970915130:84 , srv1130047613:85 , srv1403363011:85 , srv1477421584:85 , srv1693080769:85 , srv1881763717:85 , srv1998948522:85 , srv347226327:85 , srv625839411:85 , srv655504846:85 , srv867233101:684 }
2024-01-07 10:52:51,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:51,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv260117115=58, srv1418655453=23, srv1541850672=29, srv1120224247=4, srv1818168062=45, srv91502798=98, srv592093075=73, srv625839411=74, srv742439258=85, srv402176303=64, srv2144597750=55, srv1718218976=40, srv1477421584=26, srv970915130=99, srv492583965=69, srv65981265=79, srv427336737=65, srv1289099111=17, srv1640709060=32, srv482837601=67, srv1881763717=48, srv1492228624=27, srv1079262860=2, srv261210313=59, srv401316066=63, srv897645221=97, srv490954144=68, srv707637768=82, srv1015004694=0, srv893441323=96, srv227760643=56, srv1398772473=21, srv339698696=61, srv1184396574=9, srv868217043=95, srv1130047613=6, srv1203989242=11, srv1438827097=25, srv755558156=87, srv1222642441=12, srv550899740=72, srv1149239791=7, srv1642721647=33, srv867233101=94, srv801515188=88, srv746577497=86, srv1239098064=14, srv166700373=34, srv1710676532=39, srv524566156=70, srv1039959680=1, srv626057155=75, srv655504846=78, srv1122395035=5, srv1829103831=46, srv1773610557=42, srv2006081050=51, srv1286032871=16, srv1809960439=44, srv696301785=81, srv265806244=60, srv1425541511=24, srv1325646896=20, srv821306737=90, srv17784065=43, srv711015885=83, srv1097109054=3, srv1403363011=22, srv1226953303=13, srv1862210839=47, srv1998948522=50, srv2142112483=54, srv1697595955=38, srv802559383=89, srv1239373370=15, srv655368522=77, srv1759412053=41, srv840613855=92, srv1556795381=30, srv1187379230=10, srv631993028=76, srv1293365565=18, srv1676797220=35, srv857701243=93, srv347226327=62, srv1618933639=31, srv472689019=66, srv1311387210=19, srv838303189=91, srv729791928=84, srv68140221=80, srv1941237508=49, srv1154506474=8, srv2041411728=52, srv525593896=71, srv1677768500=36, srv1693080769=37, srv1498480289=28, srv2076399713=53, srv258469760=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:51,627 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:51,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:51,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-01-07 10:52:51,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-01-07 10:52:51,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-01-07 10:52:51,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-01-07 10:52:51,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-01-07 10:52:51,741 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,742 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1120224247:83 , srv1498480289:83 , srv1541850672:83 , srv1618933639:83 , srv1640709060:83 , srv1829103831:83 , srv742439258:83 , srv801515188:83 , srv857701243:83 , srv1015004694:84 , srv1039959680:84 , srv1079262860:84 , srv1097109054:84 , srv1122395035:84 , srv1149239791:84 , srv1154506474:84 , srv1184396574:84 , srv1187379230:84 , srv1203989242:84 , srv1222642441:84 , srv1226953303:84 , srv1239098064:84 , srv1239373370:84 , srv1286032871:84 , srv1289099111:84 , srv1293365565:84 , srv1311387210:84 , srv1325646896:84 , srv1398772473:84 , srv1418655453:84 , srv1425541511:84 , srv1438827097:84 , srv1492228624:84 , srv1556795381:84 , srv1642721647:84 , srv166700373:84 , srv1676797220:84 , srv1677768500:84 , srv1697595955:84 , srv1710676532:84 , srv1718218976:84 , srv1759412053:84 , srv1773610557:84 , srv17784065:84 , srv1809960439:84 , srv1818168062:84 , srv1862210839:84 , srv1941237508:84 , srv2006081050:84 , srv2041411728:84 , srv2076399713:84 , srv2142112483:84 , srv2144597750:84 , srv227760643:84 , srv258469760:84 , srv260117115:84 , srv261210313:84 , srv265806244:84 , srv339698696:84 , srv401316066:84 , srv402176303:84 , srv427336737:84 , srv472689019:84 , srv482837601:84 , srv490954144:84 , srv492583965:84 , srv524566156:84 , srv525593896:84 , srv550899740:84 , srv592093075:84 , srv626057155:84 , srv631993028:84 , srv655368522:84 , srv65981265:84 , srv68140221:84 , srv696301785:84 , srv707637768:84 , srv711015885:84 , srv729791928:84 , srv746577497:84 , srv755558156:84 , srv802559383:84 , srv821306737:84 , srv838303189:84 , srv840613855:84 , srv868217043:84 , srv893441323:84 , srv897645221:84 , srv91502798:84 , srv970915130:84 , srv1130047613:85 , srv1403363011:85 , srv1477421584:85 , srv1693080769:85 , srv1881763717:85 , srv1998948522:85 , srv347226327:85 , srv625839411:85 , srv655504846:85 , srv867233101:684 }
2024-01-07 10:52:51,764 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=438 (was 438), ProcessCount=174 (was 174), AvailableMemoryMB=6823 (was 7528)
2024-01-07 10:52:51,771 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=438, ProcessCount=174, AvailableMemoryMB=6823
2024-01-07 10:52:51,772 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-01-07 10:52:51,772 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-01-07 10:52:51,780 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv104250616:84 , srv1229681667:84 , srv1251199723:84 , srv1305197775:84 , srv1334093036:84 , srv134510346:84 , srv1399932742:84 , srv1417589198:84 , srv1428104012:84 , srv1465757440:84 , srv1472815153:84 , srv1511712721:84 , srv1599257601:84 , srv1641771347:84 , srv1693497207:84 , srv1717329618:84 , srv1747871047:84 , srv1761657106:84 , srv194795056:84 , srv1980214715:84 , srv2052922904:84 , srv311791908:84 , srv403192744:84 , srv491803472:84 , srv596790636:84 , srv63426594:84 , srv644239476:84 , srv783874480:84 , srv837315865:84 , srv792312145:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-01-07 10:52:51,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:52:51,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1693497207=14, srv1747871047=16, srv1599257601=12, srv194795056=18, srv792312145=28, srv1980214715=19, srv1305197775=3, srv491803472=23, srv1417589198=7, srv783874480=27, srv1334093036=4, srv1428104012=8, srv1229681667=1, srv403192744=22, srv837315865=29, srv1251199723=2, srv1472815153=10, srv63426594=25, srv104250616=0, srv1761657106=17, srv134510346=5, srv311791908=21, srv1465757440=9, srv1717329618=15, srv644239476=26, srv1511712721=11, srv1399932742=6, srv2052922904=20, srv596790636=24, srv1641771347=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:52:51,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:52:51,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:52:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:52:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:52:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:52:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:52:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:52:51,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-07 10:52:51,802 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.082693963529077, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:52:51,802 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912453609243155, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.082693963529077, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-01-07 10:54:51,792 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9362991 different iterations.  Found a solution that moves 1838 regions; Going from a computed imbalance of 0.9912453609243155 to a new imbalance of 4.2899836916600065E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6807407407407408, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:54:51,798 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv104250616:90 , srv1229681667:90 , srv1251199723:90 , srv1305197775:90 , srv1334093036:90 , srv134510346:90 , srv1399932742:90 , srv1417589198:90 , srv1428104012:90 , srv1465757440:90 , srv1472815153:90 , srv1511712721:90 , srv1599257601:90 , srv1641771347:90 , srv1693497207:90 , srv1717329618:90 , srv1747871047:90 , srv1761657106:90 , srv194795056:90 , srv1980214715:90 , srv2052922904:90 , srv311791908:90 , srv403192744:90 , srv491803472:90 , srv596790636:90 , srv63426594:90 , srv644239476:90 , srv783874480:90 , srv792312145:90 , srv837315865:90 }
2024-01-07 10:54:51,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-01-07 10:54:51,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1693497207=14, srv1747871047=16, srv1599257601=12, srv194795056=18, srv792312145=28, srv1980214715=19, srv1305197775=3, srv491803472=23, srv1417589198=7, srv783874480=27, srv1334093036=4, srv1428104012=8, srv1229681667=1, srv403192744=22, srv837315865=29, srv1251199723=2, srv1472815153=10, srv63426594=25, srv104250616=0, srv1761657106=17, srv134510346=5, srv311791908=21, srv1465757440=9, srv1717329618=15, srv644239476=26, srv1511712721=11, srv1399932742=6, srv2052922904=20, srv596790636=24, srv1641771347=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-01-07 10:54:51,802 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-01-07 10:54:51,803 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-01-07 10:54:51,804 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-01-07 10:54:51,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-01-07 10:54:51,814 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv104250616:90 , srv1229681667:90 , srv1251199723:90 , srv1305197775:90 , srv1334093036:90 , srv134510346:90 , srv1399932742:90 , srv1417589198:90 , srv1428104012:90 , srv1465757440:90 , srv1472815153:90 , srv1511712721:90 , srv1599257601:90 , srv1641771347:90 , srv1693497207:90 , srv1717329618:90 , srv1747871047:90 , srv1761657106:90 , srv194795056:90 , srv1980214715:90 , srv2052922904:90 , srv311791908:90 , srv403192744:90 , srv491803472:90 , srv596790636:90 , srv63426594:90 , srv644239476:90 , srv783874480:90 , srv792312145:90 , srv837315865:90 }
2024-01-07 10:54:51,826 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=391 (was 438), ProcessCount=173 (was 174), AvailableMemoryMB=5768 (was 6823)