2023-05-09 23:53:36,925 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-05-09 23:53:37,263 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-05-09 23:53:37,388 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-05-09 23:53:37,449 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-09 23:53:37,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-09 23:53:37,472 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=331, ProcessCount=172, AvailableMemoryMB=11131
2023-05-09 23:53:37,474 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-09 23:53:37,474 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-05-09 23:53:37,593 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1003872001:84 , srv1030281454:84 , srv1032872002:84 , srv1045815300:84 , srv1079001717:84 , srv1101753481:84 , srv113406588:84 , srv1187309702:84 , srv1198906854:84 , srv1206684218:84 , srv120714140:84 , srv1212407786:84 , srv1233910862:84 , srv1234925137:84 , srv1263199454:84 , srv126449567:84 , srv1407372486:84 , srv1421902098:84 , srv143493891:84 , srv1435406246:84 , srv1456333324:84 , srv1457469448:84 , srv1470950412:84 , srv1493874294:84 , srv151041972:84 , srv1524427107:84 , srv1538681584:84 , srv1559194877:84 , srv1577406983:84 , srv1606220289:84 , srv1662074649:84 , srv1665066003:84 , srv1691526765:84 , srv1703808323:84 , srv1723604346:84 , srv1757210646:84 , srv1800397093:84 , srv1863869602:84 , srv1937724357:84 , srv1941265600:84 , srv1954193297:84 , srv1986206649:84 , srv1996139067:84 , srv1999860706:84 , srv2034357462:84 , srv2048977643:84 , srv2071817983:84 , srv2078469047:84 , srv2078755867:84 , srv2084442878:84 , srv2104417878:84 , srv210778892:84 , srv2128119393:84 , srv223567315:84 , srv234714391:84 , srv267135532:84 , srv271277181:84 , srv271694946:84 , srv327399955:84 , srv331930695:84 , srv334965132:84 , srv336588857:84 , srv352054860:84 , srv393303335:84 , srv424313014:84 , srv453081022:84 , srv46860655:84 , srv476161588:84 , srv483499638:84 , srv491031894:84 , srv492995366:84 , srv497839575:84 , srv528309250:84 , srv543630898:84 , srv554565590:84 , srv560685525:84 , srv586666784:84 , srv594613442:84 , srv612751920:84 , srv627401385:84 , srv629983554:84 , srv730525940:84 , srv737946012:84 , srv740271274:84 , srv742630662:84 , srv750333400:84 , srv752750273:84 , srv772716171:84 , srv810592362:84 , srv812438376:84 , srv823357143:84 , srv834944344:84 , srv8742354:84 , srv877831979:84 , srv885973800:84 , srv891972030:84 , srv892544483:84 , srv92323618:84 , srv926941559:84 , srv815582851:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-05-09 23:53:37,602 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:53:37,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1233910862=12, srv1723604346=34, srv810592362=88, srv120714140=10, srv1079001717=4, srv1524427107=25, srv92323618=98, srv327399955=58, srv1996139067=42, srv1757210646=35, srv815582851=90, srv453081022=65, srv750333400=85, srv612751920=78, srv1234925137=13, srv1665066003=31, srv1703808323=33, srv1941265600=39, srv352054860=62, srv1101753481=5, srv336588857=61, srv1206684218=9, srv2078755867=48, srv8742354=93, srv113406588=6, srv334965132=60, srv877831979=94, srv497839575=71, srv1212407786=11, srv1954193297=40, srv271277181=56, srv1456333324=20, srv234714391=54, srv1863869602=37, srv491031894=69, srv594613442=77, srv560685525=75, srv1577406983=28, srv772716171=87, srv1003872001=0, srv483499638=68, srv1800397093=36, srv885973800=95, srv627401385=79, srv1407372486=16, srv267135532=55, srv126449567=15, srv586666784=76, srv742630662=84, srv1263199454=14, srv1032872002=2, srv492995366=70, srv1606220289=29, srv1999860706=43, srv1662074649=30, srv891972030=96, srv730525940=81, srv331930695=59, srv476161588=67, srv2128119393=52, srv223567315=53, srv823357143=91, srv424313014=64, srv1493874294=23, srv1691526765=32, srv543630898=73, srv740271274=83, srv151041972=24, srv1187309702=7, srv1470950412=22, srv752750273=86, srv1435406246=19, srv2104417878=50, srv1198906854=8, srv1045815300=3, srv1421902098=17, srv393303335=63, srv143493891=18, srv812438376=89, srv926941559=99, srv2048977643=45, srv1538681584=26, srv737946012=82, srv1937724357=38, srv834944344=92, srv271694946=57, srv554565590=74, srv629983554=80, srv1559194877=27, srv2084442878=49, srv892544483=97, srv1457469448=21, srv2034357462=44, srv2078469047=47, srv210778892=51, srv1986206649=41, srv1030281454=1, srv528309250=72, srv46860655=66, srv2071817983=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:53:37,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:53:37,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:53:37,702 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-09 23:53:37,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-09 23:53:37,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-09 23:53:37,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-09 23:53:37,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-09 23:53:37,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-09 23:53:37,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-09 23:53:37,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-09 23:53:37,890 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:37,890 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-09 23:53:37,891 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-09 23:53:38,234 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 501 ms to try 5000 different iterations.  Found a solution that moves 4716 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1752809190607726. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04094896284766, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05487256944654172, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.524, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04094896284766, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17041666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:38,253 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv267135532:81 , srv1030281454:82 , srv1198906854:82 , srv1457469448:82 , srv2078755867:82 , srv1003872001:83 , srv113406588:83 , srv1206684218:83 , srv1233910862:83 , srv1435406246:83 , srv1703808323:83 , srv629983554:83 , srv730525940:83 , srv8742354:83 , srv885973800:83 , srv1079001717:84 , srv1101753481:84 , srv336588857:84 , srv393303335:84 , srv483499638:84 , srv740271274:84 , srv810592362:84 , srv834944344:84 , srv1234925137:85 , srv1407372486:85 , srv1470950412:85 , srv1559194877:85 , srv1723604346:85 , srv1937724357:85 , srv1954193297:85 , srv2104417878:85 , srv271694946:85 , srv327399955:85 , srv491031894:85 , srv528309250:85 , srv560685525:85 , srv812438376:85 , srv1187309702:86 , srv143493891:86 , srv1493874294:86 , srv1538681584:86 , srv1691526765:86 , srv1800397093:86 , srv1986206649:86 , srv2078469047:86 , srv2128119393:86 , srv334965132:86 , srv352054860:86 , srv46860655:86 , srv492995366:86 , srv543630898:86 , srv554565590:86 , srv586666784:86 , srv594613442:86 , srv612751920:86 , srv750333400:86 , srv1032872002:87 , srv1263199454:87 , srv126449567:87 , srv1421902098:87 , srv1577406983:87 , srv1606220289:87 , srv1665066003:87 , srv1996139067:87 , srv1999860706:87 , srv223567315:87 , srv497839575:87 , srv752750273:87 , srv772716171:87 , srv823357143:87 , srv1456333324:88 , srv1524427107:88 , srv1757210646:88 , srv1863869602:88 , srv2071817983:88 , srv271277181:88 , srv424313014:88 , srv627401385:88 , srv891972030:88 , srv892544483:88 , srv92323618:88 , srv120714140:89 , srv1212407786:89 , srv151041972:89 , srv331930695:89 , srv742630662:89 , srv877831979:89 , srv2034357462:90 , srv2084442878:90 , srv210778892:90 , srv234714391:90 , srv737946012:90 , srv926941559:90 , srv1045815300:91 , srv2048977643:91 , srv1662074649:92 , srv453081022:92 , srv476161588:92 , srv1941265600:93 , srv815582851:454 }
2023-05-09 23:53:38,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:53:38,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1233910862=12, srv1723604346=34, srv810592362=88, srv120714140=10, srv1079001717=4, srv1524427107=25, srv92323618=98, srv327399955=58, srv1996139067=42, srv1757210646=35, srv815582851=90, srv453081022=65, srv750333400=85, srv612751920=78, srv1234925137=13, srv1665066003=31, srv1703808323=33, srv1941265600=39, srv352054860=62, srv1101753481=5, srv336588857=61, srv1206684218=9, srv2078755867=48, srv8742354=93, srv113406588=6, srv334965132=60, srv877831979=94, srv497839575=71, srv1212407786=11, srv1954193297=40, srv271277181=56, srv1456333324=20, srv234714391=54, srv1863869602=37, srv491031894=69, srv594613442=77, srv560685525=75, srv1577406983=28, srv772716171=87, srv1003872001=0, srv483499638=68, srv1800397093=36, srv885973800=95, srv627401385=79, srv1407372486=16, srv267135532=55, srv126449567=15, srv586666784=76, srv742630662=84, srv1263199454=14, srv1032872002=2, srv492995366=70, srv1606220289=29, srv1999860706=43, srv1662074649=30, srv891972030=96, srv730525940=81, srv331930695=59, srv476161588=67, srv2128119393=52, srv223567315=53, srv823357143=91, srv424313014=64, srv1493874294=23, srv1691526765=32, srv543630898=73, srv740271274=83, srv151041972=24, srv1187309702=7, srv1470950412=22, srv752750273=86, srv1435406246=19, srv2104417878=50, srv1198906854=8, srv1045815300=3, srv1421902098=17, srv393303335=63, srv143493891=18, srv812438376=89, srv926941559=99, srv2048977643=45, srv1538681584=26, srv737946012=82, srv1937724357=38, srv834944344=92, srv271694946=57, srv554565590=74, srv629983554=80, srv1559194877=27, srv2084442878=49, srv892544483=97, srv1457469448=21, srv2034357462=44, srv2078469047=47, srv210778892=51, srv1986206649=41, srv1030281454=1, srv528309250=72, srv46860655=66, srv2071817983=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:53:38,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:53:38,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:53:38,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:53:38,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:53:38,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:53:38,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:53:38,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:53:38,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:53:38,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:53:38,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:53:38,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:53:38,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:53:38,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:53:38,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:53:38,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:53:38,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:53:38,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-09 23:53:38,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-09 23:53:38,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-09 23:53:38,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-09 23:53:38,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-09 23:53:38,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-09 23:53:38,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-09 23:53:38,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-09 23:53:38,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-09 23:53:38,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-09 23:53:38,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-09 23:53:38,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-09 23:53:38,428 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04094896284766, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05487256944654172, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04094896284766, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17041666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:38,429 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-09 23:53:38,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1752478969229763, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04094896284766, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05487256944654172, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04094896284766, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17041666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.2375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-09 23:53:38,701 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 390 ms to try 5000 different iterations.  Found a solution that moves 2598 regions; Going from a computed imbalance of 0.1752478969229763 to a new imbalance of 0.01647700555220436. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021635348167074006); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029908202238971632, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2886666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021635348167074006); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:38,712 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv210778892:79 , srv476161588:79 , srv1662074649:81 , srv453081022:81 , srv1045815300:82 , srv120714140:82 , srv1212407786:82 , srv1941265600:82 , srv1996139067:82 , srv2048977643:82 , srv2071817983:82 , srv2084442878:82 , srv271277181:82 , srv424313014:82 , srv737946012:82 , srv92323618:82 , srv1032872002:83 , srv151041972:83 , srv1757210646:83 , srv1863869602:83 , srv223567315:83 , srv331930695:83 , srv926941559:83 , srv1263199454:84 , srv126449567:84 , srv1421902098:84 , srv1456333324:84 , srv1606220289:84 , srv2034357462:84 , srv234714391:84 , srv492995366:84 , srv586666784:84 , srv627401385:84 , srv750333400:84 , srv752750273:84 , srv892544483:84 , srv1234925137:85 , srv1577406983:85 , srv1665066003:85 , srv2128119393:85 , srv352054860:85 , srv491031894:85 , srv543630898:85 , srv823357143:85 , srv891972030:85 , srv1233910862:86 , srv143493891:86 , srv1691526765:86 , srv1954193297:86 , srv2078469047:86 , srv327399955:86 , srv334965132:86 , srv46860655:86 , srv497839575:86 , srv730525940:86 , srv740271274:86 , srv772716171:86 , srv810592362:86 , srv812438376:86 , srv877831979:86 , srv1030281454:87 , srv1723604346:87 , srv1800397093:87 , srv1937724357:87 , srv1999860706:87 , srv2104417878:87 , srv560685525:87 , srv612751920:87 , srv1003872001:88 , srv1079001717:88 , srv1407372486:88 , srv1470950412:88 , srv1493874294:88 , srv1524427107:88 , srv1538681584:88 , srv1986206649:88 , srv271694946:88 , srv393303335:88 , srv554565590:88 , srv629983554:88 , srv742630662:88 , srv885973800:88 , srv1101753481:89 , srv113406588:89 , srv1187309702:89 , srv1435406246:89 , srv1559194877:89 , srv1703808323:89 , srv336588857:89 , srv1198906854:90 , srv1206684218:90 , srv2078755867:90 , srv267135532:90 , srv483499638:90 , srv528309250:90 , srv8742354:90 , srv1457469448:91 , srv594613442:92 , srv834944344:92 , srv815582851:512 }
2023-05-09 23:53:38,717 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:53:38,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1233910862=12, srv1723604346=34, srv810592362=88, srv120714140=10, srv1079001717=4, srv1524427107=25, srv92323618=98, srv327399955=58, srv1996139067=42, srv1757210646=35, srv815582851=90, srv453081022=65, srv750333400=85, srv612751920=78, srv1234925137=13, srv1665066003=31, srv1703808323=33, srv1941265600=39, srv352054860=62, srv1101753481=5, srv336588857=61, srv1206684218=9, srv2078755867=48, srv8742354=93, srv113406588=6, srv334965132=60, srv877831979=94, srv497839575=71, srv1212407786=11, srv1954193297=40, srv271277181=56, srv1456333324=20, srv234714391=54, srv1863869602=37, srv491031894=69, srv594613442=77, srv560685525=75, srv1577406983=28, srv772716171=87, srv1003872001=0, srv483499638=68, srv1800397093=36, srv885973800=95, srv627401385=79, srv1407372486=16, srv267135532=55, srv126449567=15, srv586666784=76, srv742630662=84, srv1263199454=14, srv1032872002=2, srv492995366=70, srv1606220289=29, srv1999860706=43, srv1662074649=30, srv891972030=96, srv730525940=81, srv331930695=59, srv476161588=67, srv2128119393=52, srv223567315=53, srv823357143=91, srv424313014=64, srv1493874294=23, srv1691526765=32, srv543630898=73, srv740271274=83, srv151041972=24, srv1187309702=7, srv1470950412=22, srv752750273=86, srv1435406246=19, srv2104417878=50, srv1198906854=8, srv1045815300=3, srv1421902098=17, srv393303335=63, srv143493891=18, srv812438376=89, srv926941559=99, srv2048977643=45, srv1538681584=26, srv737946012=82, srv1937724357=38, srv834944344=92, srv271694946=57, srv554565590=74, srv629983554=80, srv1559194877=27, srv2084442878=49, srv892544483=97, srv1457469448=21, srv2034357462=44, srv2078469047=47, srv210778892=51, srv1986206649=41, srv1030281454=1, srv528309250=72, srv46860655=66, srv2071817983=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:53:38,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:53:38,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:53:38,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:53:38,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:53:38,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:53:38,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-09 23:53:38,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-09 23:53:38,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-09 23:53:38,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-09 23:53:38,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-09 23:53:38,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-09 23:53:38,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-09 23:53:38,907 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021635348167074006); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029908202238971632, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021635348167074006); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:38,907 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-09 23:53:38,907 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016458813967387822, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021635348167074006); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029908202238971632, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021635348167074006); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-09 23:53:39,231 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 460 ms to try 5000 different iterations.  Found a solution that moves 1018 regions; Going from a computed imbalance of 0.016458813967387822 to a new imbalance of 5.545375030983663E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005879447357921312); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005318160234783104); MoveCostFunction : (multiplier=7.0, imbalance=0.1131111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005879447357921312); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:39,234 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv594613442:80 , srv742630662:81 , srv834944344:81 , srv1524427107:82 , srv1187309702:83 , srv1538681584:83 , srv1559194877:83 , srv1800397093:83 , srv1941265600:83 , srv528309250:83 , srv554565590:83 , srv877831979:83 , srv1101753481:84 , srv1457469448:84 , srv1470950412:84 , srv1493874294:84 , srv1986206649:84 , srv1999860706:84 , srv2034357462:84 , srv2048977643:84 , srv2078469047:84 , srv234714391:84 , srv453081022:84 , srv483499638:84 , srv612751920:84 , srv891972030:84 , srv1045815300:85 , srv1079001717:85 , srv113406588:85 , srv1198906854:85 , srv1407372486:85 , srv143493891:85 , srv1456333324:85 , srv151041972:85 , srv1577406983:85 , srv1662074649:85 , srv1665066003:85 , srv1691526765:85 , srv1703808323:85 , srv1863869602:85 , srv1937724357:85 , srv2078755867:85 , srv2104417878:85 , srv271694946:85 , srv336588857:85 , srv46860655:85 , srv497839575:85 , srv627401385:85 , srv772716171:85 , srv8742354:85 , srv885973800:85 , srv892544483:85 , srv926941559:85 , srv1206684218:86 , srv1263199454:86 , srv1421902098:86 , srv1435406246:86 , srv1723604346:86 , srv1954193297:86 , srv2084442878:86 , srv2128119393:86 , srv267135532:86 , srv331930695:86 , srv352054860:86 , srv393303335:86 , srv476161588:86 , srv560685525:86 , srv629983554:86 , srv737946012:86 , srv752750273:86 , srv812438376:86 , srv823357143:86 , srv1003872001:87 , srv120714140:87 , srv1234925137:87 , srv126449567:87 , srv1606220289:87 , srv1757210646:87 , srv2071817983:87 , srv223567315:87 , srv271277181:87 , srv327399955:87 , srv334965132:87 , srv492995366:87 , srv543630898:87 , srv740271274:87 , srv750333400:87 , srv810592362:87 , srv1030281454:88 , srv1032872002:88 , srv1212407786:88 , srv1233910862:88 , srv424313014:88 , srv491031894:88 , srv586666784:88 , srv730525940:88 , srv1996139067:89 , srv210778892:89 , srv92323618:89 , srv815582851:544 }
2023-05-09 23:53:39,240 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:53:39,241 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1233910862=12, srv1723604346=34, srv810592362=88, srv120714140=10, srv1079001717=4, srv1524427107=25, srv92323618=98, srv327399955=58, srv1996139067=42, srv1757210646=35, srv815582851=90, srv453081022=65, srv750333400=85, srv612751920=78, srv1234925137=13, srv1665066003=31, srv1703808323=33, srv1941265600=39, srv352054860=62, srv1101753481=5, srv336588857=61, srv1206684218=9, srv2078755867=48, srv8742354=93, srv113406588=6, srv334965132=60, srv877831979=94, srv497839575=71, srv1212407786=11, srv1954193297=40, srv271277181=56, srv1456333324=20, srv234714391=54, srv1863869602=37, srv491031894=69, srv594613442=77, srv560685525=75, srv1577406983=28, srv772716171=87, srv1003872001=0, srv483499638=68, srv1800397093=36, srv885973800=95, srv627401385=79, srv1407372486=16, srv267135532=55, srv126449567=15, srv586666784=76, srv742630662=84, srv1263199454=14, srv1032872002=2, srv492995366=70, srv1606220289=29, srv1999860706=43, srv1662074649=30, srv891972030=96, srv730525940=81, srv331930695=59, srv476161588=67, srv2128119393=52, srv223567315=53, srv823357143=91, srv424313014=64, srv1493874294=23, srv1691526765=32, srv543630898=73, srv740271274=83, srv151041972=24, srv1187309702=7, srv1470950412=22, srv752750273=86, srv1435406246=19, srv2104417878=50, srv1198906854=8, srv1045815300=3, srv1421902098=17, srv393303335=63, srv143493891=18, srv812438376=89, srv926941559=99, srv2048977643=45, srv1538681584=26, srv737946012=82, srv1937724357=38, srv834944344=92, srv271694946=57, srv554565590=74, srv629983554=80, srv1559194877=27, srv2084442878=49, srv892544483=97, srv1457469448=21, srv2034357462=44, srv2078469047=47, srv210778892=51, srv1986206649=41, srv1030281454=1, srv528309250=72, srv46860655=66, srv2071817983=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:53:39,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:53:39,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:53:39,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:53:39,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:53:39,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:53:39,252 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:53:39,253 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:53:39,254 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-09 23:53:39,255 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-09 23:53:39,256 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-09 23:53:39,257 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-09 23:53:39,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-09 23:53:39,259 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-09 23:53:39,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-09 23:53:39,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-09 23:53:39,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-09 23:53:39,381 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005879447357921312); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005318160234783104); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005879447357921312); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:39,382 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-05-09 23:53:39,382 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.474093147445416E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005879447357921312); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005318160234783104); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005879447357921312); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-05-09 23:53:39,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 485 ms to try 5000 different iterations.  Found a solution that moves 229 regions; Going from a computed imbalance of 5.474093147445416E-4 to a new imbalance of 1.9517000882373303E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.025444444444444443, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:39,764 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1206684218:83 , srv1212407786:83 , srv1996139067:83 , srv334965132:83 , srv424313014:83 , srv92323618:83 , srv1003872001:84 , srv1030281454:84 , srv1032872002:84 , srv1187309702:84 , srv120714140:84 , srv1407372486:84 , srv143493891:84 , srv1435406246:84 , srv1470950412:84 , srv1493874294:84 , srv1524427107:84 , srv1606220289:84 , srv1662074649:84 , srv1691526765:84 , srv1723604346:84 , srv1757210646:84 , srv1986206649:84 , srv1999860706:84 , srv2034357462:84 , srv2084442878:84 , srv210778892:84 , srv234714391:84 , srv327399955:84 , srv331930695:84 , srv336588857:84 , srv46860655:84 , srv483499638:84 , srv491031894:84 , srv543630898:84 , srv560685525:84 , srv586666784:84 , srv594613442:84 , srv730525940:84 , srv737946012:84 , srv742630662:84 , srv772716171:84 , srv823357143:84 , srv8742354:84 , srv877831979:84 , srv926941559:84 , srv1045815300:85 , srv1079001717:85 , srv1101753481:85 , srv113406588:85 , srv1198906854:85 , srv1233910862:85 , srv1234925137:85 , srv1263199454:85 , srv126449567:85 , srv1421902098:85 , srv1456333324:85 , srv1457469448:85 , srv151041972:85 , srv1538681584:85 , srv1559194877:85 , srv1577406983:85 , srv1665066003:85 , srv1703808323:85 , srv1937724357:85 , srv1941265600:85 , srv1954193297:85 , srv2048977643:85 , srv2071817983:85 , srv2078755867:85 , srv2104417878:85 , srv2128119393:85 , srv223567315:85 , srv267135532:85 , srv271277181:85 , srv271694946:85 , srv352054860:85 , srv393303335:85 , srv453081022:85 , srv476161588:85 , srv492995366:85 , srv497839575:85 , srv528309250:85 , srv554565590:85 , srv612751920:85 , srv627401385:85 , srv740271274:85 , srv750333400:85 , srv752750273:85 , srv810592362:85 , srv812438376:85 , srv834944344:85 , srv891972030:85 , srv892544483:85 , srv1800397093:86 , srv1863869602:86 , srv2078469047:86 , srv629983554:86 , srv885973800:86 , srv815582851:632 }
2023-05-09 23:53:39,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:53:39,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1233910862=12, srv1723604346=34, srv810592362=88, srv120714140=10, srv1079001717=4, srv1524427107=25, srv92323618=98, srv327399955=58, srv1996139067=42, srv1757210646=35, srv815582851=90, srv453081022=65, srv750333400=85, srv612751920=78, srv1234925137=13, srv1665066003=31, srv1703808323=33, srv1941265600=39, srv352054860=62, srv1101753481=5, srv336588857=61, srv1206684218=9, srv2078755867=48, srv8742354=93, srv113406588=6, srv334965132=60, srv877831979=94, srv497839575=71, srv1212407786=11, srv1954193297=40, srv271277181=56, srv1456333324=20, srv234714391=54, srv1863869602=37, srv491031894=69, srv594613442=77, srv560685525=75, srv1577406983=28, srv772716171=87, srv1003872001=0, srv483499638=68, srv1800397093=36, srv885973800=95, srv627401385=79, srv1407372486=16, srv267135532=55, srv126449567=15, srv586666784=76, srv742630662=84, srv1263199454=14, srv1032872002=2, srv492995366=70, srv1606220289=29, srv1999860706=43, srv1662074649=30, srv891972030=96, srv730525940=81, srv331930695=59, srv476161588=67, srv2128119393=52, srv223567315=53, srv823357143=91, srv424313014=64, srv1493874294=23, srv1691526765=32, srv543630898=73, srv740271274=83, srv151041972=24, srv1187309702=7, srv1470950412=22, srv752750273=86, srv1435406246=19, srv2104417878=50, srv1198906854=8, srv1045815300=3, srv1421902098=17, srv393303335=63, srv143493891=18, srv812438376=89, srv926941559=99, srv2048977643=45, srv1538681584=26, srv737946012=82, srv1937724357=38, srv834944344=92, srv271694946=57, srv554565590=74, srv629983554=80, srv1559194877=27, srv2084442878=49, srv892544483=97, srv1457469448=21, srv2034357462=44, srv2078469047=47, srv210778892=51, srv1986206649=41, srv1030281454=1, srv528309250=72, srv46860655=66, srv2071817983=46} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:53:39,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:53:39,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:53:39,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:53:39,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-05-09 23:53:39,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-05-09 23:53:39,791 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-05-09 23:53:39,792 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-05-09 23:53:39,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-05-09 23:53:39,794 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-05-09 23:53:39,795 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-05-09 23:53:39,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-05-09 23:53:39,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-05-09 23:53:39,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-05-09 23:53:39,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-05-09 23:53:39,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7913508610245763E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:39,947 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1206684218:83 , srv1212407786:83 , srv1996139067:83 , srv334965132:83 , srv424313014:83 , srv92323618:83 , srv1003872001:84 , srv1030281454:84 , srv1032872002:84 , srv1187309702:84 , srv120714140:84 , srv1407372486:84 , srv143493891:84 , srv1435406246:84 , srv1470950412:84 , srv1493874294:84 , srv1524427107:84 , srv1606220289:84 , srv1662074649:84 , srv1691526765:84 , srv1723604346:84 , srv1757210646:84 , srv1986206649:84 , srv1999860706:84 , srv2034357462:84 , srv2084442878:84 , srv210778892:84 , srv234714391:84 , srv327399955:84 , srv331930695:84 , srv336588857:84 , srv46860655:84 , srv483499638:84 , srv491031894:84 , srv543630898:84 , srv560685525:84 , srv586666784:84 , srv594613442:84 , srv730525940:84 , srv737946012:84 , srv742630662:84 , srv772716171:84 , srv823357143:84 , srv8742354:84 , srv877831979:84 , srv926941559:84 , srv1045815300:85 , srv1079001717:85 , srv1101753481:85 , srv113406588:85 , srv1198906854:85 , srv1233910862:85 , srv1234925137:85 , srv1263199454:85 , srv126449567:85 , srv1421902098:85 , srv1456333324:85 , srv1457469448:85 , srv151041972:85 , srv1538681584:85 , srv1559194877:85 , srv1577406983:85 , srv1665066003:85 , srv1703808323:85 , srv1937724357:85 , srv1941265600:85 , srv1954193297:85 , srv2048977643:85 , srv2071817983:85 , srv2078755867:85 , srv2104417878:85 , srv2128119393:85 , srv223567315:85 , srv267135532:85 , srv271277181:85 , srv271694946:85 , srv352054860:85 , srv393303335:85 , srv453081022:85 , srv476161588:85 , srv492995366:85 , srv497839575:85 , srv528309250:85 , srv554565590:85 , srv612751920:85 , srv627401385:85 , srv740271274:85 , srv750333400:85 , srv752750273:85 , srv810592362:85 , srv812438376:85 , srv834944344:85 , srv891972030:85 , srv892544483:85 , srv1800397093:86 , srv1863869602:86 , srv2078469047:86 , srv629983554:86 , srv885973800:86 , srv815582851:632 }
2023-05-09 23:53:39,991 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=331 (was 331), ProcessCount=172 (was 172), AvailableMemoryMB=10340 (was 11131)
2023-05-09 23:53:40,000 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=331, ProcessCount=172, AvailableMemoryMB=10338
2023-05-09 23:53:40,001 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-05-09 23:53:40,001 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-05-09 23:53:40,012 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1036536767:84 , srv1098432570:84 , srv1106757180:84 , srv1153644803:84 , srv1388236768:84 , srv1445728346:84 , srv1606100449:84 , srv1877526644:84 , srv1946007606:84 , srv199754277:84 , srv2029531771:84 , srv2047822639:84 , srv28458327:84 , srv323925099:84 , srv33892313:84 , srv339108122:84 , srv359239813:84 , srv404981586:84 , srv407261785:84 , srv447928392:84 , srv452614645:84 , srv648538379:84 , srv673782465:84 , srv726043677:84 , srv761611652:84 , srv768380860:84 , srv790945730:84 , srv798963277:84 , srv960404473:84 , srv1115661144:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-05-09 23:53:40,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:53:40,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1445728346=6, srv33892313=15, srv339108122=16, srv1606100449=7, srv768380860=26, srv1877526644=8, srv1036536767=0, srv452614645=21, srv1115661144=3, srv1153644803=4, srv798963277=28, srv323925099=14, srv1946007606=9, srv673782465=23, srv199754277=10, srv28458327=13, srv761611652=25, srv790945730=27, srv960404473=29, srv404981586=18, srv726043677=24, srv447928392=20, srv1388236768=5, srv2047822639=12, srv1098432570=1, srv2029531771=11, srv1106757180=2, srv359239813=17, srv407261785=19, srv648538379=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:53:40,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:53:40,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:53:40,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-09 23:53:40,036 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1680147553906188, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:53:40,037 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912722452272329, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1680147553906188, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-05-09 23:55:40,027 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6738173 different iterations.  Found a solution that moves 1844 regions; Going from a computed imbalance of 0.9912722452272329 to a new imbalance of 4.303987990979898E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.682962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:55:40,034 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1036536767:90 , srv1098432570:90 , srv1106757180:90 , srv1115661144:90 , srv1153644803:90 , srv1388236768:90 , srv1445728346:90 , srv1606100449:90 , srv1877526644:90 , srv1946007606:90 , srv199754277:90 , srv2029531771:90 , srv2047822639:90 , srv28458327:90 , srv323925099:90 , srv33892313:90 , srv339108122:90 , srv359239813:90 , srv404981586:90 , srv407261785:90 , srv447928392:90 , srv452614645:90 , srv648538379:90 , srv673782465:90 , srv726043677:90 , srv761611652:90 , srv768380860:90 , srv790945730:90 , srv798963277:90 , srv960404473:90 }
2023-05-09 23:55:40,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-05-09 23:55:40,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1445728346=6, srv33892313=15, srv339108122=16, srv1606100449=7, srv768380860=26, srv1877526644=8, srv1036536767=0, srv452614645=21, srv1115661144=3, srv1153644803=4, srv798963277=28, srv323925099=14, srv1946007606=9, srv673782465=23, srv199754277=10, srv28458327=13, srv761611652=25, srv790945730=27, srv960404473=29, srv404981586=18, srv726043677=24, srv447928392=20, srv1388236768=5, srv2047822639=12, srv1098432570=1, srv2029531771=11, srv1106757180=2, srv359239813=17, srv407261785=19, srv648538379=22} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-05-09 23:55:40,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-05-09 23:55:40,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-05-09 23:55:40,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-05-09 23:55:40,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-05-09 23:55:40,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-05-09 23:55:40,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-05-09 23:55:40,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-05-09 23:55:40,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-05-09 23:55:40,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-05-09 23:55:40,067 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-05-09 23:55:40,067 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1036536767:90 , srv1098432570:90 , srv1106757180:90 , srv1115661144:90 , srv1153644803:90 , srv1388236768:90 , srv1445728346:90 , srv1606100449:90 , srv1877526644:90 , srv1946007606:90 , srv199754277:90 , srv2029531771:90 , srv2047822639:90 , srv28458327:90 , srv323925099:90 , srv33892313:90 , srv339108122:90 , srv359239813:90 , srv404981586:90 , srv407261785:90 , srv447928392:90 , srv452614645:90 , srv648538379:90 , srv673782465:90 , srv726043677:90 , srv761611652:90 , srv768380860:90 , srv790945730:90 , srv798963277:90 , srv960404473:90 }
2023-05-09 23:55:40,083 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=244 (was 331), ProcessCount=169 (was 172), AvailableMemoryMB=10034 (was 10338)