2023-02-23 05:53:58,910 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-23 05:53:59,275 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-23 05:53:59,399 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-23 05:53:59,463 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-23 05:53:59,471 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-23 05:53:59,484 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=299, ProcessCount=172, AvailableMemoryMB=7957
2023-02-23 05:53:59,487 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-23 05:53:59,487 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-23 05:53:59,611 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000040452:84 , srv1023234236:84 , srv1099479154:84 , srv1164162081:84 , srv1173697436:84 , srv1188673989:84 , srv1222258036:84 , srv1284426528:84 , srv130513034:84 , srv1335632731:84 , srv1358600896:84 , srv1380209689:84 , srv1420083343:84 , srv1429884088:84 , srv1432324239:84 , srv1439953319:84 , srv1462281111:84 , srv1462679243:84 , srv1479015527:84 , srv1497772647:84 , srv1501949886:84 , srv1509064437:84 , srv1562615224:84 , srv1580697084:84 , srv1587638354:84 , srv1604926329:84 , srv1628492541:84 , srv1656658315:84 , srv167057777:84 , srv1682443755:84 , srv1747732663:84 , srv1752702985:84 , srv1758090758:84 , srv1758467083:84 , srv1771001148:84 , srv1794701425:84 , srv1799370571:84 , srv1800255993:84 , srv1807061676:84 , srv1813835370:84 , srv1814284436:84 , srv186923969:84 , srv1879285754:84 , srv1934119915:84 , srv1959553295:84 , srv2001944409:84 , srv2008366128:84 , srv2009443451:84 , srv2009926674:84 , srv2050167383:84 , srv2082230950:84 , srv2101249117:84 , srv2145951010:84 , srv233863358:84 , srv259579011:84 , srv264949489:84 , srv28498498:84 , srv294767873:84 , srv295487490:84 , srv3111435:84 , srv312448241:84 , srv318089210:84 , srv352511960:84 , srv404537604:84 , srv418351919:84 , srv427212982:84 , srv434301424:84 , srv482262297:84 , srv505285453:84 , srv511115881:84 , srv51786412:84 , srv525370260:84 , srv595698654:84 , srv601090463:84 , srv605615232:84 , srv636052717:84 , srv638495925:84 , srv668548156:84 , srv695409957:84 , srv722215409:84 , srv742917027:84 , srv752507479:84 , srv755589899:84 , srv765580434:84 , srv767736221:84 , srv78682291:84 , srv795454953:84 , srv804267988:84 , srv817665404:84 , srv829659356:84 , srv849830408:84 , srv885594666:84 , srv885734158:84 , srv905601430:84 , srv909203026:84 , srv916596896:84 , srv922010680:84 , srv965511240:84 , srv983480421:84 , srv75399173:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-23 05:53:59,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:53:59,625 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv511115881=69, srv1628492541=26, srv130513034=8, srv264949489=55, srv312448241=60, srv427212982=65, srv1188673989=5, srv767736221=85, srv804267988=88, srv167057777=28, srv505285453=68, srv1799370571=36, srv1758467083=33, srv482262297=67, srv78682291=86, srv922010680=97, srv885734158=93, srv1580697084=23, srv2009443451=47, srv595698654=72, srv829659356=90, srv2050167383=49, srv722215409=79, srv1604926329=25, srv1752702985=31, srv1879285754=42, srv233863358=53, srv695409957=78, srv909203026=95, srv2101249117=51, srv2145951010=52, srv1587638354=24, srv418351919=64, srv817665404=89, srv1439953319=15, srv404537604=63, srv1959553295=44, srv1173697436=4, srv186923969=41, srv752507479=81, srv1771001148=34, srv294767873=57, srv1747732663=30, srv983480421=99, srv1420083343=12, srv1758090758=32, srv1358600896=10, srv1497772647=19, srv1284426528=7, srv755589899=83, srv1000040452=0, srv1335632731=9, srv849830408=91, srv965511240=98, srv352511960=62, srv3111435=59, srv259579011=54, srv434301424=66, srv1164162081=3, srv1479015527=18, srv1429884088=13, srv1682443755=29, srv28498498=56, srv1794701425=35, srv605615232=74, srv916596896=96, srv2008366128=46, srv905601430=94, srv1509064437=21, srv795454953=87, srv1222258036=6, srv1813835370=39, srv1814284436=40, srv1023234236=1, srv638495925=76, srv525370260=71, srv318089210=61, srv1501949886=20, srv1432324239=14, srv636052717=75, srv1099479154=2, srv1462679243=17, srv2001944409=45, srv885594666=92, srv1462281111=16, srv1934119915=43, srv2009926674=48, srv765580434=84, srv1656658315=27, srv75399173=82, srv51786412=70, srv668548156=77, srv2082230950=50, srv742917027=80, srv601090463=73, srv1562615224=22, srv295487490=58, srv1380209689=11, srv1800255993=37, srv1807061676=38} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:53:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:53:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:53:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:53:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:53:59,710 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:53:59,711 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:53:59,712 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 05:53:59,713 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 05:53:59,714 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 05:53:59,715 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 05:53:59,716 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 05:53:59,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 05:53:59,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 05:53:59,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 05:53:59,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:53:59,933 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 05:53:59,933 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 05:54:00,261 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 507 ms to try 5000 different iterations.  Found a solution that moves 4727 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17746422454660954. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03948445671381695, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.051452526799265944, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5252222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03948445671381695, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17275, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:00,273 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1429884088:82 , srv1934119915:82 , srv259579011:82 , srv294767873:82 , srv318089210:82 , srv965511240:82 , srv1284426528:83 , srv1771001148:83 , srv1813835370:83 , srv2145951010:83 , srv404537604:83 , srv752507479:83 , srv1188673989:84 , srv1439953319:84 , srv1501949886:84 , srv1752702985:84 , srv186923969:84 , srv51786412:84 , srv525370260:84 , srv601090463:84 , srv742917027:84 , srv849830408:84 , srv916596896:84 , srv922010680:84 , srv1173697436:85 , srv1497772647:85 , srv1628492541:85 , srv167057777:85 , srv28498498:85 , srv482262297:85 , srv638495925:85 , srv765580434:85 , srv78682291:85 , srv795454953:85 , srv1023234236:86 , srv1099479154:86 , srv1164162081:86 , srv1509064437:86 , srv1580697084:86 , srv1799370571:86 , srv1814284436:86 , srv1959553295:86 , srv2001944409:86 , srv2009443451:86 , srv2101249117:86 , srv511115881:86 , srv668548156:86 , srv1335632731:87 , srv1358600896:87 , srv1380209689:87 , srv1462281111:87 , srv1587638354:87 , srv1604926329:87 , srv1758090758:87 , srv1794701425:87 , srv2082230950:87 , srv264949489:87 , srv595698654:87 , srv636052717:87 , srv695409957:87 , srv722215409:87 , srv817665404:87 , srv885594666:87 , srv1222258036:88 , srv130513034:88 , srv1420083343:88 , srv1462679243:88 , srv1479015527:88 , srv1656658315:88 , srv1747732663:88 , srv1758467083:88 , srv1807061676:88 , srv2050167383:88 , srv3111435:88 , srv312448241:88 , srv352511960:88 , srv427212982:88 , srv505285453:88 , srv605615232:88 , srv885734158:88 , srv905601430:88 , srv983480421:88 , srv1562615224:89 , srv1800255993:89 , srv295487490:89 , srv418351919:89 , srv434301424:89 , srv755589899:89 , srv767736221:89 , srv804267988:89 , srv829659356:89 , srv909203026:89 , srv1432324239:90 , srv233863358:90 , srv1000040452:91 , srv2009926674:91 , srv1682443755:92 , srv1879285754:92 , srv2008366128:93 , srv75399173:441 }
2023-02-23 05:54:00,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:54:00,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv511115881=69, srv1628492541=26, srv130513034=8, srv264949489=55, srv312448241=60, srv427212982=65, srv1188673989=5, srv767736221=85, srv804267988=88, srv167057777=28, srv505285453=68, srv1799370571=36, srv1758467083=33, srv482262297=67, srv78682291=86, srv922010680=97, srv885734158=93, srv1580697084=23, srv2009443451=47, srv595698654=72, srv829659356=90, srv2050167383=49, srv722215409=79, srv1604926329=25, srv1752702985=31, srv1879285754=42, srv233863358=53, srv695409957=78, srv909203026=95, srv2101249117=51, srv2145951010=52, srv1587638354=24, srv418351919=64, srv817665404=89, srv1439953319=15, srv404537604=63, srv1959553295=44, srv1173697436=4, srv186923969=41, srv752507479=81, srv1771001148=34, srv294767873=57, srv1747732663=30, srv983480421=99, srv1420083343=12, srv1758090758=32, srv1358600896=10, srv1497772647=19, srv1284426528=7, srv755589899=83, srv1000040452=0, srv1335632731=9, srv849830408=91, srv965511240=98, srv352511960=62, srv3111435=59, srv259579011=54, srv434301424=66, srv1164162081=3, srv1479015527=18, srv1429884088=13, srv1682443755=29, srv28498498=56, srv1794701425=35, srv605615232=74, srv916596896=96, srv2008366128=46, srv905601430=94, srv1509064437=21, srv795454953=87, srv1222258036=6, srv1813835370=39, srv1814284436=40, srv1023234236=1, srv638495925=76, srv525370260=71, srv318089210=61, srv1501949886=20, srv1432324239=14, srv636052717=75, srv1099479154=2, srv1462679243=17, srv2001944409=45, srv885594666=92, srv1462281111=16, srv1934119915=43, srv2009926674=48, srv765580434=84, srv1656658315=27, srv75399173=82, srv51786412=70, srv668548156=77, srv2082230950=50, srv742917027=80, srv601090463=73, srv1562615224=22, srv295487490=58, srv1380209689=11, srv1800255993=37, srv1807061676=38} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:54:00,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:54:00,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:54:00,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 05:54:00,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 05:54:00,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 05:54:00,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 05:54:00,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 05:54:00,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 05:54:00,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 05:54:00,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 05:54:00,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 05:54:00,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 05:54:00,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03948445671381695, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.051452526799265944, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03948445671381695, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17275, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:00,422 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 05:54:00,422 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.177431125385167, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03948445671381695, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.051452526799265944, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03948445671381695, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17275, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23866666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 05:54:00,674 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 362 ms to try 5000 different iterations.  Found a solution that moves 2638 regions; Going from a computed imbalance of 0.177431125385167 to a new imbalance of 0.014606629596644787. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020375573256582395); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032349105806260645, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2931111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020375573256582395); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0435, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:00,688 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2008366128:79 , srv2009926674:79 , srv1682443755:80 , srv1432324239:81 , srv1562615224:81 , srv1879285754:81 , srv983480421:81 , srv1747732663:82 , srv233863358:82 , srv295487490:82 , srv418351919:82 , srv505285453:82 , srv804267988:82 , srv829659356:82 , srv1000040452:83 , srv1656658315:83 , srv1758090758:83 , srv427212982:83 , srv434301424:83 , srv605615232:83 , srv722215409:83 , srv755589899:83 , srv909203026:83 , srv1420083343:84 , srv1462679243:84 , srv1479015527:84 , srv1800255993:84 , srv2050167383:84 , srv3111435:84 , srv767736221:84 , srv1099479154:85 , srv1335632731:85 , srv1462281111:85 , srv1587638354:85 , srv1604926329:85 , srv1758467083:85 , srv1807061676:85 , srv2101249117:85 , srv28498498:85 , srv695409957:85 , srv885594666:85 , srv905601430:85 , srv1023234236:86 , srv130513034:86 , srv1380209689:86 , srv1497772647:86 , srv1509064437:86 , srv1628492541:86 , srv1794701425:86 , srv1799370571:86 , srv1959553295:86 , srv2001944409:86 , srv352511960:86 , srv511115881:86 , srv595698654:86 , srv636052717:86 , srv765580434:86 , srv78682291:86 , srv1173697436:87 , srv1358600896:87 , srv1580697084:87 , srv1752702985:87 , srv1813835370:87 , srv186923969:87 , srv2009443451:87 , srv2082230950:87 , srv482262297:87 , srv525370260:87 , srv601090463:87 , srv638495925:87 , srv742917027:87 , srv885734158:87 , srv916596896:87 , srv1164162081:88 , srv1188673989:88 , srv1222258036:88 , srv1439953319:88 , srv1771001148:88 , srv1814284436:88 , srv264949489:88 , srv668548156:88 , srv795454953:88 , srv922010680:88 , srv1501949886:89 , srv294767873:89 , srv312448241:89 , srv318089210:89 , srv51786412:89 , srv817665404:89 , srv849830408:89 , srv965511240:89 , srv1284426528:90 , srv1429884088:90 , srv167057777:90 , srv2145951010:90 , srv404537604:90 , srv752507479:90 , srv1934119915:91 , srv259579011:91 , srv75399173:514 }
2023-02-23 05:54:00,692 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:54:00,693 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv511115881=69, srv1628492541=26, srv130513034=8, srv264949489=55, srv312448241=60, srv427212982=65, srv1188673989=5, srv767736221=85, srv804267988=88, srv167057777=28, srv505285453=68, srv1799370571=36, srv1758467083=33, srv482262297=67, srv78682291=86, srv922010680=97, srv885734158=93, srv1580697084=23, srv2009443451=47, srv595698654=72, srv829659356=90, srv2050167383=49, srv722215409=79, srv1604926329=25, srv1752702985=31, srv1879285754=42, srv233863358=53, srv695409957=78, srv909203026=95, srv2101249117=51, srv2145951010=52, srv1587638354=24, srv418351919=64, srv817665404=89, srv1439953319=15, srv404537604=63, srv1959553295=44, srv1173697436=4, srv186923969=41, srv752507479=81, srv1771001148=34, srv294767873=57, srv1747732663=30, srv983480421=99, srv1420083343=12, srv1758090758=32, srv1358600896=10, srv1497772647=19, srv1284426528=7, srv755589899=83, srv1000040452=0, srv1335632731=9, srv849830408=91, srv965511240=98, srv352511960=62, srv3111435=59, srv259579011=54, srv434301424=66, srv1164162081=3, srv1479015527=18, srv1429884088=13, srv1682443755=29, srv28498498=56, srv1794701425=35, srv605615232=74, srv916596896=96, srv2008366128=46, srv905601430=94, srv1509064437=21, srv795454953=87, srv1222258036=6, srv1813835370=39, srv1814284436=40, srv1023234236=1, srv638495925=76, srv525370260=71, srv318089210=61, srv1501949886=20, srv1432324239=14, srv636052717=75, srv1099479154=2, srv1462679243=17, srv2001944409=45, srv885594666=92, srv1462281111=16, srv1934119915=43, srv2009926674=48, srv765580434=84, srv1656658315=27, srv75399173=82, srv51786412=70, srv668548156=77, srv2082230950=50, srv742917027=80, srv601090463=73, srv1562615224=22, srv295487490=58, srv1380209689=11, srv1800255993=37, srv1807061676=38} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:54:00,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:54:00,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:54:00,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:54:00,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:54:00,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:54:00,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:54:00,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:54:00,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:54:00,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:54:00,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:54:00,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:54:00,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 05:54:00,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 05:54:00,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 05:54:00,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 05:54:00,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 05:54:00,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 05:54:00,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 05:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 05:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 05:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 05:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 05:54:00,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 05:54:00,858 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020375573256582395); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032349105806260645, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020375573256582395); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0435, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:00,858 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 05:54:00,858 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014588157925841849, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020375573256582395); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.032349105806260645, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020375573256582395); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0435, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 05:54:01,084 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 1009 regions; Going from a computed imbalance of 0.014588157925841849 to a new imbalance of 5.551645595234833E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005427250157295044); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0059553143325186565); MoveCostFunction : (multiplier=7.0, imbalance=0.1121111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005427250157295044); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:01,086 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv312448241:81 , srv817665404:81 , srv1222258036:82 , srv167057777:82 , srv1164162081:83 , srv130513034:83 , srv1358600896:83 , srv264949489:83 , srv885734158:83 , srv1000040452:84 , srv1580697084:84 , srv1758467083:84 , srv1794701425:84 , srv1814284436:84 , srv2082230950:84 , srv2145951010:84 , srv259579011:84 , srv352511960:84 , srv51786412:84 , srv638495925:84 , srv668548156:84 , srv849830408:84 , srv905601430:84 , srv1023234236:85 , srv1284426528:85 , srv1335632731:85 , srv1380209689:85 , srv1420083343:85 , srv1429884088:85 , srv1462679243:85 , srv1479015527:85 , srv1587638354:85 , srv1604926329:85 , srv1800255993:85 , srv1879285754:85 , srv1934119915:85 , srv1959553295:85 , srv2001944409:85 , srv2009443451:85 , srv404537604:85 , srv434301424:85 , srv482262297:85 , srv511115881:85 , srv595698654:85 , srv636052717:85 , srv752507479:85 , srv755589899:85 , srv767736221:85 , srv795454953:85 , srv885594666:85 , srv1099479154:86 , srv1173697436:86 , srv1188673989:86 , srv1432324239:86 , srv1439953319:86 , srv1462281111:86 , srv1501949886:86 , srv1509064437:86 , srv1656658315:86 , srv1682443755:86 , srv1752702985:86 , srv1771001148:86 , srv1799370571:86 , srv1807061676:86 , srv2008366128:86 , srv2050167383:86 , srv233863358:86 , srv28498498:86 , srv3111435:86 , srv318089210:86 , srv418351919:86 , srv427212982:86 , srv525370260:86 , srv601090463:86 , srv605615232:86 , srv695409957:86 , srv765580434:86 , srv829659356:86 , srv909203026:86 , srv922010680:86 , srv965511240:86 , srv1497772647:87 , srv1628492541:87 , srv1747732663:87 , srv1758090758:87 , srv186923969:87 , srv2009926674:87 , srv2101249117:87 , srv294767873:87 , srv295487490:87 , srv505285453:87 , srv742917027:87 , srv78682291:87 , srv804267988:87 , srv916596896:87 , srv1562615224:88 , srv722215409:88 , srv983480421:88 , srv1813835370:89 , srv75399173:551 }
2023-02-23 05:54:01,089 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:54:01,090 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv511115881=69, srv1628492541=26, srv130513034=8, srv264949489=55, srv312448241=60, srv427212982=65, srv1188673989=5, srv767736221=85, srv804267988=88, srv167057777=28, srv505285453=68, srv1799370571=36, srv1758467083=33, srv482262297=67, srv78682291=86, srv922010680=97, srv885734158=93, srv1580697084=23, srv2009443451=47, srv595698654=72, srv829659356=90, srv2050167383=49, srv722215409=79, srv1604926329=25, srv1752702985=31, srv1879285754=42, srv233863358=53, srv695409957=78, srv909203026=95, srv2101249117=51, srv2145951010=52, srv1587638354=24, srv418351919=64, srv817665404=89, srv1439953319=15, srv404537604=63, srv1959553295=44, srv1173697436=4, srv186923969=41, srv752507479=81, srv1771001148=34, srv294767873=57, srv1747732663=30, srv983480421=99, srv1420083343=12, srv1758090758=32, srv1358600896=10, srv1497772647=19, srv1284426528=7, srv755589899=83, srv1000040452=0, srv1335632731=9, srv849830408=91, srv965511240=98, srv352511960=62, srv3111435=59, srv259579011=54, srv434301424=66, srv1164162081=3, srv1479015527=18, srv1429884088=13, srv1682443755=29, srv28498498=56, srv1794701425=35, srv605615232=74, srv916596896=96, srv2008366128=46, srv905601430=94, srv1509064437=21, srv795454953=87, srv1222258036=6, srv1813835370=39, srv1814284436=40, srv1023234236=1, srv638495925=76, srv525370260=71, srv318089210=61, srv1501949886=20, srv1432324239=14, srv636052717=75, srv1099479154=2, srv1462679243=17, srv2001944409=45, srv885594666=92, srv1462281111=16, srv1934119915=43, srv2009926674=48, srv765580434=84, srv1656658315=27, srv75399173=82, srv51786412=70, srv668548156=77, srv2082230950=50, srv742917027=80, srv601090463=73, srv1562615224=22, srv295487490=58, srv1380209689=11, srv1800255993=37, srv1807061676=38} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:54:01,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:54:01,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 05:54:01,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 05:54:01,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 05:54:01,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 05:54:01,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 05:54:01,215 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005427250157295044); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0059553143325186565); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005427250157295044); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:01,215 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-23 05:54:01,215 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.480993905165981E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005427250157295044); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0059553143325186565); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005427250157295044); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-23 05:54:01,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 454 ms to try 5000 different iterations.  Found a solution that moves 199 regions; Going from a computed imbalance of 5.480993905165981E-4 to a new imbalance of 1.9389785011401796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.022111111111111113); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:01,573 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1807061676:82 , srv1501949886:83 , srv1628492541:83 , srv1813835370:83 , srv1879285754:83 , srv1000040452:84 , srv1173697436:84 , srv1188673989:84 , srv1222258036:84 , srv1284426528:84 , srv1358600896:84 , srv1380209689:84 , srv1439953319:84 , srv1462281111:84 , srv1497772647:84 , srv1509064437:84 , srv1562615224:84 , srv1799370571:84 , srv1814284436:84 , srv186923969:84 , srv1934119915:84 , srv2008366128:84 , srv2009443451:84 , srv2050167383:84 , srv2082230950:84 , srv2101249117:84 , srv233863358:84 , srv259579011:84 , srv264949489:84 , srv294767873:84 , srv295487490:84 , srv3111435:84 , srv352511960:84 , srv404537604:84 , srv595698654:84 , srv636052717:84 , srv668548156:84 , srv695409957:84 , srv722215409:84 , srv742917027:84 , srv752507479:84 , srv767736221:84 , srv78682291:84 , srv795454953:84 , srv804267988:84 , srv909203026:84 , srv916596896:84 , srv922010680:84 , srv1023234236:85 , srv1099479154:85 , srv1164162081:85 , srv130513034:85 , srv1335632731:85 , srv1420083343:85 , srv1429884088:85 , srv1432324239:85 , srv1462679243:85 , srv1479015527:85 , srv1580697084:85 , srv1587638354:85 , srv1604926329:85 , srv1656658315:85 , srv167057777:85 , srv1682443755:85 , srv1747732663:85 , srv1752702985:85 , srv1758090758:85 , srv1758467083:85 , srv1771001148:85 , srv1794701425:85 , srv1800255993:85 , srv1959553295:85 , srv2001944409:85 , srv2009926674:85 , srv2145951010:85 , srv312448241:85 , srv318089210:85 , srv418351919:85 , srv427212982:85 , srv434301424:85 , srv482262297:85 , srv505285453:85 , srv511115881:85 , srv51786412:85 , srv525370260:85 , srv605615232:85 , srv755589899:85 , srv765580434:85 , srv817665404:85 , srv829659356:85 , srv849830408:85 , srv885594666:85 , srv885734158:85 , srv905601430:85 , srv965511240:85 , srv983480421:85 , srv28498498:86 , srv601090463:86 , srv638495925:86 , srv75399173:636 }
2023-02-23 05:54:01,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:54:01,576 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv511115881=69, srv1628492541=26, srv130513034=8, srv264949489=55, srv312448241=60, srv427212982=65, srv1188673989=5, srv767736221=85, srv804267988=88, srv167057777=28, srv505285453=68, srv1799370571=36, srv1758467083=33, srv482262297=67, srv78682291=86, srv922010680=97, srv885734158=93, srv1580697084=23, srv2009443451=47, srv595698654=72, srv829659356=90, srv2050167383=49, srv722215409=79, srv1604926329=25, srv1752702985=31, srv1879285754=42, srv233863358=53, srv695409957=78, srv909203026=95, srv2101249117=51, srv2145951010=52, srv1587638354=24, srv418351919=64, srv817665404=89, srv1439953319=15, srv404537604=63, srv1959553295=44, srv1173697436=4, srv186923969=41, srv752507479=81, srv1771001148=34, srv294767873=57, srv1747732663=30, srv983480421=99, srv1420083343=12, srv1758090758=32, srv1358600896=10, srv1497772647=19, srv1284426528=7, srv755589899=83, srv1000040452=0, srv1335632731=9, srv849830408=91, srv965511240=98, srv352511960=62, srv3111435=59, srv259579011=54, srv434301424=66, srv1164162081=3, srv1479015527=18, srv1429884088=13, srv1682443755=29, srv28498498=56, srv1794701425=35, srv605615232=74, srv916596896=96, srv2008366128=46, srv905601430=94, srv1509064437=21, srv795454953=87, srv1222258036=6, srv1813835370=39, srv1814284436=40, srv1023234236=1, srv638495925=76, srv525370260=71, srv318089210=61, srv1501949886=20, srv1432324239=14, srv636052717=75, srv1099479154=2, srv1462679243=17, srv2001944409=45, srv885594666=92, srv1462281111=16, srv1934119915=43, srv2009926674=48, srv765580434=84, srv1656658315=27, srv75399173=82, srv51786412=70, srv668548156=77, srv2082230950=50, srv742917027=80, srv601090463=73, srv1562615224=22, srv295487490=58, srv1380209689=11, srv1800255993=37, srv1807061676=38} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:54:01,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-23 05:54:01,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-23 05:54:01,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-23 05:54:01,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-23 05:54:01,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-23 05:54:01,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-23 05:54:01,700 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7996357229072622E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:01,700 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1807061676:82 , srv1501949886:83 , srv1628492541:83 , srv1813835370:83 , srv1879285754:83 , srv1000040452:84 , srv1173697436:84 , srv1188673989:84 , srv1222258036:84 , srv1284426528:84 , srv1358600896:84 , srv1380209689:84 , srv1439953319:84 , srv1462281111:84 , srv1497772647:84 , srv1509064437:84 , srv1562615224:84 , srv1799370571:84 , srv1814284436:84 , srv186923969:84 , srv1934119915:84 , srv2008366128:84 , srv2009443451:84 , srv2050167383:84 , srv2082230950:84 , srv2101249117:84 , srv233863358:84 , srv259579011:84 , srv264949489:84 , srv294767873:84 , srv295487490:84 , srv3111435:84 , srv352511960:84 , srv404537604:84 , srv595698654:84 , srv636052717:84 , srv668548156:84 , srv695409957:84 , srv722215409:84 , srv742917027:84 , srv752507479:84 , srv767736221:84 , srv78682291:84 , srv795454953:84 , srv804267988:84 , srv909203026:84 , srv916596896:84 , srv922010680:84 , srv1023234236:85 , srv1099479154:85 , srv1164162081:85 , srv130513034:85 , srv1335632731:85 , srv1420083343:85 , srv1429884088:85 , srv1432324239:85 , srv1462679243:85 , srv1479015527:85 , srv1580697084:85 , srv1587638354:85 , srv1604926329:85 , srv1656658315:85 , srv167057777:85 , srv1682443755:85 , srv1747732663:85 , srv1752702985:85 , srv1758090758:85 , srv1758467083:85 , srv1771001148:85 , srv1794701425:85 , srv1800255993:85 , srv1959553295:85 , srv2001944409:85 , srv2009926674:85 , srv2145951010:85 , srv312448241:85 , srv318089210:85 , srv418351919:85 , srv427212982:85 , srv434301424:85 , srv482262297:85 , srv505285453:85 , srv511115881:85 , srv51786412:85 , srv525370260:85 , srv605615232:85 , srv755589899:85 , srv765580434:85 , srv817665404:85 , srv829659356:85 , srv849830408:85 , srv885594666:85 , srv885734158:85 , srv905601430:85 , srv965511240:85 , srv983480421:85 , srv28498498:86 , srv601090463:86 , srv638495925:86 , srv75399173:636 }
2023-02-23 05:54:01,731 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=364 (was 299) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=7466 (was 7957)
2023-02-23 05:54:01,741 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=364, ProcessCount=172, AvailableMemoryMB=7465
2023-02-23 05:54:01,742 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-23 05:54:01,742 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-23 05:54:01,753 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1052290629:84 , srv1057932238:84 , srv1088191368:84 , srv11239021:84 , srv1157016771:84 , srv1495339084:84 , srv1566823602:84 , srv1717726072:84 , srv1718249973:84 , srv1821613772:84 , srv2106997126:84 , srv2124542583:84 , srv309400582:84 , srv318115489:84 , srv401046404:84 , srv409366508:84 , srv430195481:84 , srv467599443:84 , srv487747928:84 , srv563829418:84 , srv6971189:84 , srv710480117:84 , srv722808312:84 , srv73593082:84 , srv820736743:84 , srv840981726:84 , srv854239932:84 , srv866440002:84 , srv94224602:84 , srv1380652669:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-23 05:54:01,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:54:01,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv318115489=14, srv1157016771=4, srv722808312=23, srv11239021=3, srv487747928=19, srv854239932=27, srv866440002=28, srv467599443=18, srv94224602=29, srv1088191368=2, srv1717726072=8, srv1052290629=0, srv2124542583=12, srv820736743=25, srv1718249973=9, srv563829418=20, srv2106997126=11, srv73593082=24, srv1566823602=7, srv401046404=15, srv409366508=16, srv6971189=21, srv1380652669=5, srv1821613772=10, srv1057932238=1, srv309400582=13, srv710480117=22, srv840981726=26, srv430195481=17, srv1495339084=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:54:01,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:54:01,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:54:01,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-23 05:54:01,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1297534125939726, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:54:01,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912601892030525, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1297534125939726, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-23 05:56:01,775 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6363211 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912601892030525 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:56:01,781 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1052290629:90 , srv1057932238:90 , srv1088191368:90 , srv11239021:90 , srv1157016771:90 , srv1380652669:90 , srv1495339084:90 , srv1566823602:90 , srv1717726072:90 , srv1718249973:90 , srv1821613772:90 , srv2106997126:90 , srv2124542583:90 , srv309400582:90 , srv318115489:90 , srv401046404:90 , srv409366508:90 , srv430195481:90 , srv467599443:90 , srv487747928:90 , srv563829418:90 , srv6971189:90 , srv710480117:90 , srv722808312:90 , srv73593082:90 , srv820736743:90 , srv840981726:90 , srv854239932:90 , srv866440002:90 , srv94224602:90 }
2023-02-23 05:56:01,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-23 05:56:01,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv318115489=14, srv1157016771=4, srv722808312=23, srv11239021=3, srv487747928=19, srv854239932=27, srv866440002=28, srv467599443=18, srv94224602=29, srv1088191368=2, srv1717726072=8, srv1052290629=0, srv2124542583=12, srv820736743=25, srv1718249973=9, srv563829418=20, srv2106997126=11, srv73593082=24, srv1566823602=7, srv401046404=15, srv409366508=16, srv6971189=21, srv1380652669=5, srv1821613772=10, srv1057932238=1, srv309400582=13, srv710480117=22, srv840981726=26, srv430195481=17, srv1495339084=6} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-23 05:56:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-23 05:56:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-23 05:56:01,798 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-23 05:56:01,799 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1052290629:90 , srv1057932238:90 , srv1088191368:90 , srv11239021:90 , srv1157016771:90 , srv1380652669:90 , srv1495339084:90 , srv1566823602:90 , srv1717726072:90 , srv1718249973:90 , srv1821613772:90 , srv2106997126:90 , srv2124542583:90 , srv309400582:90 , srv318115489:90 , srv401046404:90 , srv409366508:90 , srv430195481:90 , srv467599443:90 , srv487747928:90 , srv563829418:90 , srv6971189:90 , srv710480117:90 , srv722808312:90 , srv73593082:90 , srv820736743:90 , srv840981726:90 , srv854239932:90 , srv866440002:90 , srv94224602:90 }
2023-02-23 05:56:01,812 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=180 (was 364), ProcessCount=172 (was 172), AvailableMemoryMB=7221 (was 7465)