2023-05-29 17:53:23,945 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-29 17:53:24,225 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-29 17:53:24,329 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-29 17:53:24,391 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-29 17:53:24,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-29 17:53:24,416 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=172, AvailableMemoryMB=4735
2023-05-29 17:53:24,417 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-29 17:53:24,418 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-29 17:53:24,541 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1015262924:84 , srv1050694449:84 , srv1059650268:84 , srv1065503929:84 , srv1074069459:84 , srv1079943542:84 , srv1082817418:84 , srv1090668575:84 , srv1158321033:84 , srv1163782649:84 , srv1170090881:84 , srv1179893368:84 , srv1191121846:84 , srv1216811934:84 , srv1231541341:84 , srv1274184523:84 , srv1276645600:84 , srv1286452433:84 , srv1317056052:84 , srv1353399617:84 , srv1362248206:84 , srv1368335156:84 , srv1377830582:84 , srv13973371:84 , srv1456178109:84 , srv1461435382:84 , srv1483733428:84 , srv153755627:84 , srv1551853351:84 , srv155274814:84 , srv1554884407:84 , srv1601195594:84 , srv1650952257:84 , srv1654776581:84 , srv1661059991:84 , srv1686871008:84 , srv1698289485:84 , srv1712006973:84 , srv1718042772:84 , srv1736206654:84 , srv1742520664:84 , srv1794257535:84 , srv1797137808:84 , srv1859188282:84 , srv1880802147:84 , srv1884506910:84 , srv1906522553:84 , srv1909831574:84 , srv1976422493:84 , srv1984019433:84 , srv1987080434:84 , srv1999210562:84 , srv2016672612:84 , srv2017900456:84 , srv2076269355:84 , srv2089246907:84 , srv2109727515:84 , srv2134152412:84 , srv227171005:84 , srv236898452:84 , srv264439284:84 , srv289180154:84 , srv312819397:84 , srv351180615:84 , srv388174621:84 , srv430451438:84 , srv482500643:84 , srv489152999:84 , srv510438641:84 , srv517127439:84 , srv532959056:84 , srv550861805:84 , srv58645026:84 , srv595807064:84 , srv596422962:84 , srv611642768:84 , srv622744746:84 , srv63185724:84 , srv671443854:84 , srv706526074:84 , srv714175312:84 , srv718678326:84 , srv758477329:84 , srv760723016:84 , srv773053369:84 , srv785772006:84 , srv788117399:84 , srv792530533:84 , srv805928780:84 , srv817794357:84 , srv878000781:84 , srv878179524:84 , srv883726408:84 , srv912343090:84 , srv948477441:84 , srv956746005:84 , srv983307215:84 , srv988863432:84 , srv989095675:84 , srv2142876147:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-29 17:53:24,554 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:53:24,562 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1362248206=20, srv788117399=87, srv1859188282=43, srv1884506910=45, srv2134152412=57, srv817794357=90, srv785772006=86, srv1074069459=4, srv1274184523=15, srv312819397=63, srv227171005=59, srv956746005=96, srv58645026=73, srv1377830582=22, srv878179524=92, srv1059650268=2, srv1353399617=19, srv883726408=93, srv718678326=82, srv1158321033=8, srv1698289485=36, srv1712006973=37, srv1090668575=7, srv1650952257=32, srv2142876147=58, srv2017900456=53, srv289180154=62, srv595807064=74, srv153755627=27, srv236898452=60, srv989095675=99, srv805928780=89, srv912343090=94, srv1231541341=14, srv983307215=97, srv878000781=91, srv988863432=98, srv2089246907=55, srv1718042772=38, srv510438641=69, srv1456178109=24, srv1461435382=25, srv760723016=84, srv1976422493=48, srv1216811934=13, srv1483733428=26, srv1880802147=44, srv948477441=95, srv155274814=29, srv2076269355=54, srv482500643=67, srv1065503929=3, srv1797137808=42, srv1601195594=31, srv1050694449=1, srv1163782649=9, srv1079943542=5, srv1999210562=51, srv351180615=64, srv532959056=71, srv1906522553=46, srv706526074=80, srv1742520664=40, srv1686871008=35, srv2109727515=56, srv792530533=88, srv489152999=68, srv758477329=83, srv1179893368=11, srv714175312=81, srv1286452433=17, srv13973371=23, srv430451438=66, srv671443854=79, srv1551853351=28, srv1654776581=33, srv622744746=77, srv1909831574=47, srv1736206654=39, srv1170090881=10, srv1191121846=12, srv1661059991=34, srv1276645600=16, srv596422962=75, srv1317056052=18, srv1082817418=6, srv1794257535=41, srv517127439=70, srv63185724=78, srv1368335156=21, srv264439284=61, srv1015262924=0, srv1987080434=50, srv550861805=72, srv611642768=76, srv773053369=85, srv1554884407=30, srv2016672612=52, srv1984019433=49, srv388174621=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:53:24,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:53:24,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:53:24,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-29 17:53:24,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-29 17:53:24,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-29 17:53:24,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-29 17:53:24,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-29 17:53:24,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-29 17:53:24,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-29 17:53:24,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-29 17:53:24,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-29 17:53:24,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:24,778 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-29 17:53:24,778 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-29 17:53:25,109 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 452 ms to try 5000 different iterations.  Found a solution that moves 4699 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17605956821591726. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039486667463145775, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05611036103109016, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5221111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039486667463145775, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17116666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:25,120 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1698289485:82 , srv1742520664:82 , srv1999210562:82 , srv948477441:82 , srv1686871008:83 , srv1976422493:83 , srv2109727515:83 , srv236898452:83 , srv289180154:83 , srv611642768:83 , srv1216811934:84 , srv1274184523:84 , srv1551853351:84 , srv1712006973:84 , srv482500643:84 , srv489152999:84 , srv517127439:84 , srv706526074:84 , srv805928780:84 , srv956746005:84 , srv1065503929:85 , srv1276645600:85 , srv1286452433:85 , srv1317056052:85 , srv1353399617:85 , srv1368335156:85 , srv1377830582:85 , srv1483733428:85 , srv1718042772:85 , srv1794257535:85 , srv1984019433:85 , srv2076269355:85 , srv227171005:85 , srv264439284:85 , srv351180615:85 , srv58645026:85 , srv622744746:85 , srv817794357:85 , srv912343090:85 , srv988863432:85 , srv1158321033:86 , srv1179893368:86 , srv1461435382:86 , srv1859188282:86 , srv1880802147:86 , srv388174621:86 , srv596422962:86 , srv714175312:86 , srv792530533:86 , srv883726408:86 , srv983307215:86 , srv1050694449:87 , srv1059650268:87 , srv1079943542:87 , srv1082817418:87 , srv1090668575:87 , srv1170090881:87 , srv1231541341:87 , srv13973371:87 , srv155274814:87 , srv1909831574:87 , srv1987080434:87 , srv2017900456:87 , srv2134152412:87 , srv312819397:87 , srv63185724:87 , srv671443854:87 , srv718678326:87 , srv760723016:87 , srv878000781:87 , srv878179524:87 , srv1654776581:88 , srv1661059991:88 , srv1736206654:88 , srv2089246907:88 , srv430451438:88 , srv550861805:88 , srv785772006:88 , srv788117399:88 , srv989095675:88 , srv1015262924:89 , srv1191121846:89 , srv153755627:89 , srv1554884407:89 , srv1650952257:89 , srv2016672612:89 , srv595807064:89 , srv1601195594:90 , srv1884506910:90 , srv510438641:90 , srv758477329:90 , srv773053369:90 , srv1362248206:91 , srv532959056:91 , srv1074069459:92 , srv1163782649:92 , srv1456178109:92 , srv1797137808:92 , srv1906522553:92 , srv2142876147:441 }
2023-05-29 17:53:25,124 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:53:25,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1362248206=20, srv788117399=87, srv1859188282=43, srv1884506910=45, srv2134152412=57, srv817794357=90, srv785772006=86, srv1074069459=4, srv1274184523=15, srv312819397=63, srv227171005=59, srv956746005=96, srv58645026=73, srv1377830582=22, srv878179524=92, srv1059650268=2, srv1353399617=19, srv883726408=93, srv718678326=82, srv1158321033=8, srv1698289485=36, srv1712006973=37, srv1090668575=7, srv1650952257=32, srv2142876147=58, srv2017900456=53, srv289180154=62, srv595807064=74, srv153755627=27, srv236898452=60, srv989095675=99, srv805928780=89, srv912343090=94, srv1231541341=14, srv983307215=97, srv878000781=91, srv988863432=98, srv2089246907=55, srv1718042772=38, srv510438641=69, srv1456178109=24, srv1461435382=25, srv760723016=84, srv1976422493=48, srv1216811934=13, srv1483733428=26, srv1880802147=44, srv948477441=95, srv155274814=29, srv2076269355=54, srv482500643=67, srv1065503929=3, srv1797137808=42, srv1601195594=31, srv1050694449=1, srv1163782649=9, srv1079943542=5, srv1999210562=51, srv351180615=64, srv532959056=71, srv1906522553=46, srv706526074=80, srv1742520664=40, srv1686871008=35, srv2109727515=56, srv792530533=88, srv489152999=68, srv758477329=83, srv1179893368=11, srv714175312=81, srv1286452433=17, srv13973371=23, srv430451438=66, srv671443854=79, srv1551853351=28, srv1654776581=33, srv622744746=77, srv1909831574=47, srv1736206654=39, srv1170090881=10, srv1191121846=12, srv1661059991=34, srv1276645600=16, srv596422962=75, srv1317056052=18, srv1082817418=6, srv1794257535=41, srv517127439=70, srv63185724=78, srv1368335156=21, srv264439284=61, srv1015262924=0, srv1987080434=50, srv550861805=72, srv611642768=76, srv773053369=85, srv1554884407=30, srv2016672612=52, srv1984019433=49, srv388174621=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:53:25,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:53:25,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:53:25,139 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:53:25,140 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:53:25,141 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:53:25,142 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-29 17:53:25,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-29 17:53:25,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-29 17:53:25,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-29 17:53:25,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-29 17:53:25,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-29 17:53:25,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-29 17:53:25,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-29 17:53:25,281 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039486667463145775, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05611036103109016, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039486667463145775, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17116666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:25,281 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-29 17:53:25,281 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17602666511466517, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039486667463145775, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05611036103109016, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039486667463145775, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17116666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-29 17:53:25,646 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 476 ms to try 5000 different iterations.  Found a solution that moves 2668 regions; Going from a computed imbalance of 0.17602666511466517 to a new imbalance of 0.014464992654783883. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022061709867428628); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03403305978382135, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29644444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022061709867428628); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045083333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:25,663 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1074069459:78 , srv1163782649:79 , srv1601195594:80 , srv758477329:80 , srv1015262924:81 , srv1456178109:81 , srv1797137808:81 , srv1906522553:81 , srv532959056:81 , srv773053369:81 , srv1362248206:82 , srv510438641:82 , srv595807064:82 , srv788117399:82 , srv13973371:83 , srv1650952257:83 , srv883726408:83 , srv989095675:83 , srv1050694449:84 , srv1059650268:84 , srv1661059991:84 , srv1736206654:84 , srv1884506910:84 , srv2017900456:84 , srv2089246907:84 , srv388174621:84 , srv430451438:84 , srv622744746:84 , srv718678326:84 , srv878179524:84 , srv1090668575:85 , srv1158321033:85 , srv1170090881:85 , srv1191121846:85 , srv1461435382:85 , srv155274814:85 , srv1554884407:85 , srv1654776581:85 , srv2016672612:85 , srv2134152412:85 , srv351180615:85 , srv550861805:85 , srv58645026:85 , srv760723016:85 , srv805928780:85 , srv878000781:85 , srv1079943542:86 , srv1082817418:86 , srv1286452433:86 , srv1353399617:86 , srv1377830582:86 , srv153755627:86 , srv1712006973:86 , srv1859188282:86 , srv1909831574:86 , srv1984019433:86 , srv1987080434:86 , srv312819397:86 , srv596422962:86 , srv63185724:86 , srv671443854:86 , srv785772006:86 , srv792530533:86 , srv912343090:86 , srv988863432:86 , srv1065503929:87 , srv1179893368:87 , srv1216811934:87 , srv1483733428:87 , srv1551853351:87 , srv1718042772:87 , srv1794257535:87 , srv1880802147:87 , srv264439284:87 , srv517127439:87 , srv817794357:87 , srv1276645600:88 , srv1317056052:88 , srv1686871008:88 , srv1698289485:88 , srv1999210562:88 , srv227171005:88 , srv236898452:88 , srv706526074:88 , srv1274184523:89 , srv1368335156:89 , srv2076269355:89 , srv2109727515:89 , srv611642768:89 , srv714175312:89 , srv983307215:89 , srv1742520664:90 , srv1976422493:90 , srv289180154:90 , srv482500643:90 , srv948477441:90 , srv956746005:90 , srv1231541341:91 , srv489152999:91 , srv2142876147:529 }
2023-05-29 17:53:25,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:53:25,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1362248206=20, srv788117399=87, srv1859188282=43, srv1884506910=45, srv2134152412=57, srv817794357=90, srv785772006=86, srv1074069459=4, srv1274184523=15, srv312819397=63, srv227171005=59, srv956746005=96, srv58645026=73, srv1377830582=22, srv878179524=92, srv1059650268=2, srv1353399617=19, srv883726408=93, srv718678326=82, srv1158321033=8, srv1698289485=36, srv1712006973=37, srv1090668575=7, srv1650952257=32, srv2142876147=58, srv2017900456=53, srv289180154=62, srv595807064=74, srv153755627=27, srv236898452=60, srv989095675=99, srv805928780=89, srv912343090=94, srv1231541341=14, srv983307215=97, srv878000781=91, srv988863432=98, srv2089246907=55, srv1718042772=38, srv510438641=69, srv1456178109=24, srv1461435382=25, srv760723016=84, srv1976422493=48, srv1216811934=13, srv1483733428=26, srv1880802147=44, srv948477441=95, srv155274814=29, srv2076269355=54, srv482500643=67, srv1065503929=3, srv1797137808=42, srv1601195594=31, srv1050694449=1, srv1163782649=9, srv1079943542=5, srv1999210562=51, srv351180615=64, srv532959056=71, srv1906522553=46, srv706526074=80, srv1742520664=40, srv1686871008=35, srv2109727515=56, srv792530533=88, srv489152999=68, srv758477329=83, srv1179893368=11, srv714175312=81, srv1286452433=17, srv13973371=23, srv430451438=66, srv671443854=79, srv1551853351=28, srv1654776581=33, srv622744746=77, srv1909831574=47, srv1736206654=39, srv1170090881=10, srv1191121846=12, srv1661059991=34, srv1276645600=16, srv596422962=75, srv1317056052=18, srv1082817418=6, srv1794257535=41, srv517127439=70, srv63185724=78, srv1368335156=21, srv264439284=61, srv1015262924=0, srv1987080434=50, srv550861805=72, srv611642768=76, srv773053369=85, srv1554884407=30, srv2016672612=52, srv1984019433=49, srv388174621=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:53:25,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:53:25,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:53:25,696 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:53:25,697 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:53:25,698 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:53:25,699 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-29 17:53:25,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-29 17:53:25,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-29 17:53:25,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-29 17:53:25,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-29 17:53:25,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-29 17:53:25,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-29 17:53:25,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-29 17:53:25,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-29 17:53:25,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-29 17:53:25,857 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022061709867428628); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03403305978382135, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022061709867428628); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045083333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:25,858 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-29 17:53:25,858 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014446310919491147, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022061709867428628); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03403305978382135, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022061709867428628); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.045083333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-29 17:53:26,172 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 419 ms to try 5000 different iterations.  Found a solution that moves 1003 regions; Going from a computed imbalance of 0.014446310919491147 to a new imbalance of 4.781355500482096E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006007436053853573); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006564879518897745); MoveCostFunction : (multiplier=7.0, imbalance=0.11144444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006007436053853573); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002916666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:26,174 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv153755627:81 , srv1231541341:82 , srv489152999:82 , srv714175312:82 , srv1368335156:83 , srv1884506910:83 , srv482500643:83 , srv983307215:83 , srv1065503929:84 , srv1079943542:84 , srv1082817418:84 , srv1274184523:84 , srv1362248206:84 , srv1554884407:84 , srv1654776581:84 , srv2016672612:84 , srv2076269355:84 , srv227171005:84 , srv289180154:84 , srv312819397:84 , srv430451438:84 , srv510438641:84 , srv63185724:84 , srv671443854:84 , srv785772006:84 , srv956746005:84 , srv1090668575:85 , srv1179893368:85 , srv1191121846:85 , srv1317056052:85 , srv1456178109:85 , srv1483733428:85 , srv1650952257:85 , srv1661059991:85 , srv1742520664:85 , srv1797137808:85 , srv1859188282:85 , srv1880802147:85 , srv1906522553:85 , srv1909831574:85 , srv1976422493:85 , srv1987080434:85 , srv2109727515:85 , srv264439284:85 , srv532959056:85 , srv550861805:85 , srv706526074:85 , srv760723016:85 , srv792530533:85 , srv817794357:85 , srv948477441:85 , srv1015262924:86 , srv1050694449:86 , srv1059650268:86 , srv1276645600:86 , srv1286452433:86 , srv1377830582:86 , srv13973371:86 , srv1461435382:86 , srv1551853351:86 , srv155274814:86 , srv1686871008:86 , srv1718042772:86 , srv1736206654:86 , srv1794257535:86 , srv1984019433:86 , srv2017900456:86 , srv2089246907:86 , srv2134152412:86 , srv517127439:86 , srv611642768:86 , srv773053369:86 , srv878000781:86 , srv878179524:86 , srv912343090:86 , srv988863432:86 , srv1074069459:87 , srv1158321033:87 , srv1163782649:87 , srv1170090881:87 , srv1216811934:87 , srv1353399617:87 , srv1601195594:87 , srv1712006973:87 , srv236898452:87 , srv388174621:87 , srv595807064:87 , srv596422962:87 , srv718678326:87 , srv758477329:87 , srv788117399:87 , srv989095675:87 , srv1698289485:88 , srv351180615:88 , srv805928780:88 , srv1999210562:89 , srv58645026:89 , srv883726408:89 , srv622744746:90 , srv2142876147:541 }
2023-05-29 17:53:26,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:53:26,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1362248206=20, srv788117399=87, srv1859188282=43, srv1884506910=45, srv2134152412=57, srv817794357=90, srv785772006=86, srv1074069459=4, srv1274184523=15, srv312819397=63, srv227171005=59, srv956746005=96, srv58645026=73, srv1377830582=22, srv878179524=92, srv1059650268=2, srv1353399617=19, srv883726408=93, srv718678326=82, srv1158321033=8, srv1698289485=36, srv1712006973=37, srv1090668575=7, srv1650952257=32, srv2142876147=58, srv2017900456=53, srv289180154=62, srv595807064=74, srv153755627=27, srv236898452=60, srv989095675=99, srv805928780=89, srv912343090=94, srv1231541341=14, srv983307215=97, srv878000781=91, srv988863432=98, srv2089246907=55, srv1718042772=38, srv510438641=69, srv1456178109=24, srv1461435382=25, srv760723016=84, srv1976422493=48, srv1216811934=13, srv1483733428=26, srv1880802147=44, srv948477441=95, srv155274814=29, srv2076269355=54, srv482500643=67, srv1065503929=3, srv1797137808=42, srv1601195594=31, srv1050694449=1, srv1163782649=9, srv1079943542=5, srv1999210562=51, srv351180615=64, srv532959056=71, srv1906522553=46, srv706526074=80, srv1742520664=40, srv1686871008=35, srv2109727515=56, srv792530533=88, srv489152999=68, srv758477329=83, srv1179893368=11, srv714175312=81, srv1286452433=17, srv13973371=23, srv430451438=66, srv671443854=79, srv1551853351=28, srv1654776581=33, srv622744746=77, srv1909831574=47, srv1736206654=39, srv1170090881=10, srv1191121846=12, srv1661059991=34, srv1276645600=16, srv596422962=75, srv1317056052=18, srv1082817418=6, srv1794257535=41, srv517127439=70, srv63185724=78, srv1368335156=21, srv264439284=61, srv1015262924=0, srv1987080434=50, srv550861805=72, srv611642768=76, srv773053369=85, srv1554884407=30, srv2016672612=52, srv1984019433=49, srv388174621=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:53:26,195 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:53:26,196 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:53:26,197 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:53:26,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-29 17:53:26,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-29 17:53:26,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-29 17:53:26,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-29 17:53:26,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-29 17:53:26,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-29 17:53:26,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-29 17:53:26,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-29 17:53:26,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-29 17:53:26,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006007436053853573); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006564879518897745); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006007436053853573); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002916666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:26,309 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-29 17:53:26,309 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.711123939392842E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006007436053853573); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006564879518897745); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006007436053853573); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.002916666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-29 17:53:26,655 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 441 ms to try 5000 different iterations.  Found a solution that moves 207 regions; Going from a computed imbalance of 4.711123939392842E-4 to a new imbalance of 1.9224001340375897E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.023); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:26,656 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1231541341:82 , srv1170090881:83 , srv1191121846:83 , srv1276645600:83 , srv1906522553:83 , srv1999210562:83 , srv58645026:83 , srv596422962:83 , srv622744746:83 , srv1158321033:84 , srv1163782649:84 , srv1179893368:84 , srv1216811934:84 , srv1286452433:84 , srv1317056052:84 , srv1353399617:84 , srv155274814:84 , srv1554884407:84 , srv1698289485:84 , srv1718042772:84 , srv1736206654:84 , srv1794257535:84 , srv1797137808:84 , srv1880802147:84 , srv1909831574:84 , srv1976422493:84 , srv1987080434:84 , srv2016672612:84 , srv2089246907:84 , srv2134152412:84 , srv236898452:84 , srv351180615:84 , srv388174621:84 , srv532959056:84 , srv550861805:84 , srv595807064:84 , srv611642768:84 , srv718678326:84 , srv785772006:84 , srv817794357:84 , srv878000781:84 , srv883726408:84 , srv912343090:84 , srv956746005:84 , srv983307215:84 , srv1050694449:85 , srv1059650268:85 , srv1074069459:85 , srv1079943542:85 , srv1082817418:85 , srv1090668575:85 , srv1274184523:85 , srv1368335156:85 , srv1377830582:85 , srv1456178109:85 , srv1461435382:85 , srv1483733428:85 , srv153755627:85 , srv1551853351:85 , srv1601195594:85 , srv1650952257:85 , srv1654776581:85 , srv1661059991:85 , srv1686871008:85 , srv1712006973:85 , srv1742520664:85 , srv1859188282:85 , srv1884506910:85 , srv1984019433:85 , srv2017900456:85 , srv2076269355:85 , srv2109727515:85 , srv227171005:85 , srv264439284:85 , srv289180154:85 , srv312819397:85 , srv482500643:85 , srv489152999:85 , srv517127439:85 , srv63185724:85 , srv671443854:85 , srv714175312:85 , srv758477329:85 , srv773053369:85 , srv788117399:85 , srv805928780:85 , srv878179524:85 , srv948477441:85 , srv988863432:85 , srv989095675:85 , srv1015262924:86 , srv1065503929:86 , srv1362248206:86 , srv13973371:86 , srv430451438:86 , srv510438641:86 , srv706526074:86 , srv760723016:86 , srv792530533:86 , srv2142876147:631 }
2023-05-29 17:53:26,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:53:26,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1362248206=20, srv788117399=87, srv1859188282=43, srv1884506910=45, srv2134152412=57, srv817794357=90, srv785772006=86, srv1074069459=4, srv1274184523=15, srv312819397=63, srv227171005=59, srv956746005=96, srv58645026=73, srv1377830582=22, srv878179524=92, srv1059650268=2, srv1353399617=19, srv883726408=93, srv718678326=82, srv1158321033=8, srv1698289485=36, srv1712006973=37, srv1090668575=7, srv1650952257=32, srv2142876147=58, srv2017900456=53, srv289180154=62, srv595807064=74, srv153755627=27, srv236898452=60, srv989095675=99, srv805928780=89, srv912343090=94, srv1231541341=14, srv983307215=97, srv878000781=91, srv988863432=98, srv2089246907=55, srv1718042772=38, srv510438641=69, srv1456178109=24, srv1461435382=25, srv760723016=84, srv1976422493=48, srv1216811934=13, srv1483733428=26, srv1880802147=44, srv948477441=95, srv155274814=29, srv2076269355=54, srv482500643=67, srv1065503929=3, srv1797137808=42, srv1601195594=31, srv1050694449=1, srv1163782649=9, srv1079943542=5, srv1999210562=51, srv351180615=64, srv532959056=71, srv1906522553=46, srv706526074=80, srv1742520664=40, srv1686871008=35, srv2109727515=56, srv792530533=88, srv489152999=68, srv758477329=83, srv1179893368=11, srv714175312=81, srv1286452433=17, srv13973371=23, srv430451438=66, srv671443854=79, srv1551853351=28, srv1654776581=33, srv622744746=77, srv1909831574=47, srv1736206654=39, srv1170090881=10, srv1191121846=12, srv1661059991=34, srv1276645600=16, srv596422962=75, srv1317056052=18, srv1082817418=6, srv1794257535=41, srv517127439=70, srv63185724=78, srv1368335156=21, srv264439284=61, srv1015262924=0, srv1987080434=50, srv550861805=72, srv611642768=76, srv773053369=85, srv1554884407=30, srv2016672612=52, srv1984019433=49, srv388174621=65} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:53:26,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:53:26,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-29 17:53:26,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-29 17:53:26,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-29 17:53:26,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-29 17:53:26,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-29 17:53:26,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-29 17:53:26,785 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.777455636076716E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:26,785 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1231541341:82 , srv1170090881:83 , srv1191121846:83 , srv1276645600:83 , srv1906522553:83 , srv1999210562:83 , srv58645026:83 , srv596422962:83 , srv622744746:83 , srv1158321033:84 , srv1163782649:84 , srv1179893368:84 , srv1216811934:84 , srv1286452433:84 , srv1317056052:84 , srv1353399617:84 , srv155274814:84 , srv1554884407:84 , srv1698289485:84 , srv1718042772:84 , srv1736206654:84 , srv1794257535:84 , srv1797137808:84 , srv1880802147:84 , srv1909831574:84 , srv1976422493:84 , srv1987080434:84 , srv2016672612:84 , srv2089246907:84 , srv2134152412:84 , srv236898452:84 , srv351180615:84 , srv388174621:84 , srv532959056:84 , srv550861805:84 , srv595807064:84 , srv611642768:84 , srv718678326:84 , srv785772006:84 , srv817794357:84 , srv878000781:84 , srv883726408:84 , srv912343090:84 , srv956746005:84 , srv983307215:84 , srv1050694449:85 , srv1059650268:85 , srv1074069459:85 , srv1079943542:85 , srv1082817418:85 , srv1090668575:85 , srv1274184523:85 , srv1368335156:85 , srv1377830582:85 , srv1456178109:85 , srv1461435382:85 , srv1483733428:85 , srv153755627:85 , srv1551853351:85 , srv1601195594:85 , srv1650952257:85 , srv1654776581:85 , srv1661059991:85 , srv1686871008:85 , srv1712006973:85 , srv1742520664:85 , srv1859188282:85 , srv1884506910:85 , srv1984019433:85 , srv2017900456:85 , srv2076269355:85 , srv2109727515:85 , srv227171005:85 , srv264439284:85 , srv289180154:85 , srv312819397:85 , srv482500643:85 , srv489152999:85 , srv517127439:85 , srv63185724:85 , srv671443854:85 , srv714175312:85 , srv758477329:85 , srv773053369:85 , srv788117399:85 , srv805928780:85 , srv878179524:85 , srv948477441:85 , srv988863432:85 , srv989095675:85 , srv1015262924:86 , srv1065503929:86 , srv1362248206:86 , srv13973371:86 , srv430451438:86 , srv510438641:86 , srv706526074:86 , srv760723016:86 , srv792530533:86 , srv2142876147:631 }
2023-05-29 17:53:26,813 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=266 (was 254) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=4237 (was 4735)
2023-05-29 17:53:26,820 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=266, ProcessCount=172, AvailableMemoryMB=4236
2023-05-29 17:53:26,821 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-29 17:53:26,821 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-29 17:53:26,833 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1010100253:84 , srv1033106554:84 , srv1037910985:84 , srv107577444:84 , srv1190750296:84 , srv1204134048:84 , srv1288809696:84 , srv150087078:84 , srv1549518303:84 , srv1770165113:84 , srv1827251014:84 , srv1966693169:84 , srv2017425897:84 , srv2031867684:84 , srv2038123400:84 , srv2062554398:84 , srv254272023:84 , srv27269246:84 , srv280151617:84 , srv351950553:84 , srv531655522:84 , srv5703143:84 , srv574074301:84 , srv671453038:84 , srv684772342:84 , srv728205330:84 , srv83122791:84 , srv837070563:84 , srv887763168:84 , srv324613052:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-29 17:53:26,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:53:26,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv324613052=19, srv887763168=29, srv728205330=26, srv1770165113=9, srv1827251014=10, srv27269246=17, srv150087078=7, srv254272023=16, srv574074301=23, srv2062554398=15, srv1010100253=0, srv5703143=22, srv684772342=25, srv1190750296=4, srv531655522=21, srv1966693169=11, srv837070563=28, srv2038123400=14, srv351950553=20, srv83122791=27, srv1549518303=8, srv671453038=24, srv1033106554=1, srv280151617=18, srv2017425897=12, srv1204134048=5, srv1288809696=6, srv2031867684=13, srv107577444=3, srv1037910985=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:53:26,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:53:26,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:53:26,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:53:26,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:53:26,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:53:26,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-29 17:53:26,873 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.154084604159844, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:53:26,873 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912678558820662, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.154084604159844, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-29 17:55:26,853 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6118899 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912678558820662 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:55:26,858 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1010100253:90 , srv1033106554:90 , srv1037910985:90 , srv107577444:90 , srv1190750296:90 , srv1204134048:90 , srv1288809696:90 , srv150087078:90 , srv1549518303:90 , srv1770165113:90 , srv1827251014:90 , srv1966693169:90 , srv2017425897:90 , srv2031867684:90 , srv2038123400:90 , srv2062554398:90 , srv254272023:90 , srv27269246:90 , srv280151617:90 , srv324613052:90 , srv351950553:90 , srv531655522:90 , srv5703143:90 , srv574074301:90 , srv671453038:90 , srv684772342:90 , srv728205330:90 , srv83122791:90 , srv837070563:90 , srv887763168:90 }
2023-05-29 17:55:26,859 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-29 17:55:26,860 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv324613052=19, srv887763168=29, srv728205330=26, srv1770165113=9, srv1827251014=10, srv27269246=17, srv150087078=7, srv254272023=16, srv574074301=23, srv2062554398=15, srv1010100253=0, srv5703143=22, srv684772342=25, srv1190750296=4, srv531655522=21, srv1966693169=11, srv837070563=28, srv2038123400=14, srv351950553=20, srv83122791=27, srv1549518303=8, srv671453038=24, srv1033106554=1, srv280151617=18, srv2017425897=12, srv1204134048=5, srv1288809696=6, srv2031867684=13, srv107577444=3, srv1037910985=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-29 17:55:26,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-29 17:55:26,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-29 17:55:26,876 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-29 17:55:26,876 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1010100253:90 , srv1033106554:90 , srv1037910985:90 , srv107577444:90 , srv1190750296:90 , srv1204134048:90 , srv1288809696:90 , srv150087078:90 , srv1549518303:90 , srv1770165113:90 , srv1827251014:90 , srv1966693169:90 , srv2017425897:90 , srv2031867684:90 , srv2038123400:90 , srv2062554398:90 , srv254272023:90 , srv27269246:90 , srv280151617:90 , srv324613052:90 , srv351950553:90 , srv531655522:90 , srv5703143:90 , srv574074301:90 , srv671453038:90 , srv684772342:90 , srv728205330:90 , srv83122791:90 , srv837070563:90 , srv887763168:90 }
2023-05-29 17:55:26,888 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=264 (was 264), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=217 (was 266), ProcessCount=168 (was 172), AvailableMemoryMB=3976 (was 4236)