2023-07-17 20:53:11,937 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-07-17 20:53:12,254 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-07-17 20:53:12,373 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-07-17 20:53:12,442 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-17 20:53:12,449 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-17 20:53:12,462 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=301, ProcessCount=172, AvailableMemoryMB=7614
2023-07-17 20:53:12,464 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-17 20:53:12,464 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-07-17 20:53:12,626 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1044814675:84 , srv1084845221:84 , srv1132720962:84 , srv1133814941:84 , srv1135458264:84 , srv1149205545:84 , srv1149472337:84 , srv1167876626:84 , srv1183514945:84 , srv1216238641:84 , srv1234884198:84 , srv1242811274:84 , srv1249551768:84 , srv1253726843:84 , srv1269257415:84 , srv1273742411:84 , srv1278253750:84 , srv1303131988:84 , srv1314295973:84 , srv1323486188:84 , srv1337960120:84 , srv1343333727:84 , srv1353871419:84 , srv1380960150:84 , srv1422477052:84 , srv1428061628:84 , srv1463005023:84 , srv1464223346:84 , srv147040010:84 , srv1472376905:84 , srv1516049960:84 , srv1529760856:84 , srv1533365032:84 , srv1551974262:84 , srv1556640497:84 , srv1571281630:84 , srv1585257851:84 , srv1617097576:84 , srv1631375250:84 , srv1633005542:84 , srv1639967867:84 , srv1665070281:84 , srv1729376307:84 , srv1774390419:84 , srv1779015521:84 , srv1796235398:84 , srv1824889112:84 , srv1848348786:84 , srv1872170029:84 , srv1930389005:84 , srv1945090755:84 , srv1963923118:84 , srv1976279153:84 , srv1997742674:84 , srv199946131:84 , srv2019813955:84 , srv2032385448:84 , srv2038673266:84 , srv2040636407:84 , srv2048685802:84 , srv2104565986:84 , srv2122350848:84 , srv229843757:84 , srv230751483:84 , srv232761688:84 , srv255998003:84 , srv259780349:84 , srv271126238:84 , srv276688374:84 , srv276830591:84 , srv301312584:84 , srv303416331:84 , srv362647944:84 , srv430398515:84 , srv457415407:84 , srv467827834:84 , srv506446051:84 , srv547158772:84 , srv597916296:84 , srv618135364:84 , srv619863180:84 , srv620526058:84 , srv631266051:84 , srv66371267:84 , srv671983390:84 , srv685104233:84 , srv691989676:84 , srv743975458:84 , srv756521473:84 , srv756924040:84 , srv773229433:84 , srv788310827:84 , srv797905465:84 , srv813394698:84 , srv903225095:84 , srv912742670:84 , srv915317931:84 , srv937442501:84 , srv956085974:84 , srv1310810894:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-07-17 20:53:12,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:53:12,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv691989676=87, srv301312584=71, srv303416331=72, srv1044814675=0, srv199946131=55, srv1639967867=41, srv1872170029=49, srv547158772=78, srv457415407=75, srv147040010=29, srv1463005023=27, srv259780349=67, srv230751483=64, srv937442501=98, srv915317931=97, srv1084845221=1, srv1343333727=22, srv797905465=93, srv1585257851=37, srv2032385448=57, srv276830591=70, srv1242811274=11, srv631266051=83, srv1132720962=2, srv2048685802=60, srv1617097576=38, srv671983390=85, srv1551974262=34, srv1631375250=39, srv1779015521=45, srv1249551768=12, srv912742670=96, srv773229433=91, srv2122350848=62, srv788310827=92, srv1976279153=53, srv1253726843=13, srv685104233=86, srv1997742674=54, srv1133814941=3, srv2104565986=61, srv255998003=66, srv1310810894=18, srv1234884198=10, srv1464223346=28, srv1848348786=48, srv1774390419=44, srv620526058=82, srv1323486188=20, srv66371267=84, srv956085974=99, srv1183514945=8, srv1380960150=24, srv1135458264=4, srv1945090755=51, srv1273742411=15, srv1529760856=32, srv1796235398=46, srv430398515=74, srv1571281630=36, srv1533365032=33, srv2038673266=58, srv276688374=69, srv1472376905=30, srv1269257415=14, srv229843757=63, srv1516049960=31, srv1353871419=23, srv1824889112=47, srv1729376307=43, srv1216238641=9, srv1303131988=17, srv362647944=73, srv1665070281=42, srv743975458=88, srv1930389005=50, srv2019813955=56, srv1278253750=16, srv1963923118=52, srv506446051=77, srv467827834=76, srv1149472337=6, srv1337960120=21, srv903225095=95, srv1149205545=5, srv232761688=65, srv1633005542=40, srv756924040=90, srv618135364=80, srv619863180=81, srv271126238=68, srv1167876626=7, srv1422477052=25, srv1428061628=26, srv2040636407=59, srv756521473=89, srv813394698=94, srv1314295973=19, srv597916296=79, srv1556640497=35} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:53:12,749 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:53:12,750 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:53:12,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:53:12,752 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:53:12,753 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-17 20:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-17 20:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-17 20:53:12,754 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-17 20:53:12,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-17 20:53:12,756 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-17 20:53:12,757 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-17 20:53:12,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-17 20:53:12,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-17 20:53:12,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-17 20:53:12,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-17 20:53:13,175 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:13,176 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-17 20:53:13,176 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-17 20:53:13,564 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 771 ms to try 5000 different iterations.  Found a solution that moves 4702 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17424024684171446. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04219149451905964, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054042008849955976, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5224444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04219149451905964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16925, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23758333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:13,601 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv597916296:80 , srv1133814941:82 , srv1216238641:82 , srv1273742411:83 , srv1353871419:83 , srv1556640497:83 , srv1872170029:83 , srv232761688:83 , srv506446051:83 , srv547158772:83 , srv619863180:83 , srv797905465:83 , srv912742670:83 , srv1149472337:84 , srv1303131988:84 , srv1314295973:84 , srv1380960150:84 , srv1422477052:84 , srv1533365032:84 , srv1639967867:84 , srv1729376307:84 , srv1779015521:84 , srv1963923118:84 , srv199946131:84 , srv2019813955:84 , srv259780349:84 , srv457415407:84 , srv671983390:84 , srv756521473:84 , srv773229433:84 , srv1167876626:85 , srv1242811274:85 , srv1253726843:85 , srv1269257415:85 , srv1428061628:85 , srv1516049960:85 , srv1585257851:85 , srv1945090755:85 , srv1976279153:85 , srv1997742674:85 , srv2040636407:85 , srv230751483:85 , srv276830591:85 , srv430398515:85 , srv788310827:85 , srv915317931:85 , srv1084845221:86 , srv1135458264:86 , srv1323486188:86 , srv1463005023:86 , srv147040010:86 , srv1529760856:86 , srv1665070281:86 , srv1774390419:86 , srv1848348786:86 , srv1930389005:86 , srv229843757:86 , srv301312584:86 , srv303416331:86 , srv618135364:86 , srv903225095:86 , srv956085974:86 , srv1278253750:87 , srv1824889112:87 , srv276688374:87 , srv362647944:87 , srv631266051:87 , srv66371267:87 , srv685104233:87 , srv743975458:87 , srv1132720962:88 , srv1337960120:88 , srv1551974262:88 , srv1571281630:88 , srv1631375250:88 , srv1633005542:88 , srv1796235398:88 , srv2048685802:88 , srv255998003:88 , srv1472376905:89 , srv2032385448:89 , srv467827834:89 , srv691989676:89 , srv1044814675:90 , srv1343333727:90 , srv1464223346:90 , srv1617097576:90 , srv271126238:90 , srv620526058:90 , srv756924040:90 , srv937442501:90 , srv1149205545:91 , srv2104565986:91 , srv2122350848:91 , srv813394698:91 , srv1183514945:92 , srv1249551768:92 , srv2038673266:92 , srv1234884198:93 , srv1310810894:465 }
2023-07-17 20:53:13,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:53:13,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv691989676=87, srv301312584=71, srv303416331=72, srv1044814675=0, srv199946131=55, srv1639967867=41, srv1872170029=49, srv547158772=78, srv457415407=75, srv147040010=29, srv1463005023=27, srv259780349=67, srv230751483=64, srv937442501=98, srv915317931=97, srv1084845221=1, srv1343333727=22, srv797905465=93, srv1585257851=37, srv2032385448=57, srv276830591=70, srv1242811274=11, srv631266051=83, srv1132720962=2, srv2048685802=60, srv1617097576=38, srv671983390=85, srv1551974262=34, srv1631375250=39, srv1779015521=45, srv1249551768=12, srv912742670=96, srv773229433=91, srv2122350848=62, srv788310827=92, srv1976279153=53, srv1253726843=13, srv685104233=86, srv1997742674=54, srv1133814941=3, srv2104565986=61, srv255998003=66, srv1310810894=18, srv1234884198=10, srv1464223346=28, srv1848348786=48, srv1774390419=44, srv620526058=82, srv1323486188=20, srv66371267=84, srv956085974=99, srv1183514945=8, srv1380960150=24, srv1135458264=4, srv1945090755=51, srv1273742411=15, srv1529760856=32, srv1796235398=46, srv430398515=74, srv1571281630=36, srv1533365032=33, srv2038673266=58, srv276688374=69, srv1472376905=30, srv1269257415=14, srv229843757=63, srv1516049960=31, srv1353871419=23, srv1824889112=47, srv1729376307=43, srv1216238641=9, srv1303131988=17, srv362647944=73, srv1665070281=42, srv743975458=88, srv1930389005=50, srv2019813955=56, srv1278253750=16, srv1963923118=52, srv506446051=77, srv467827834=76, srv1149472337=6, srv1337960120=21, srv903225095=95, srv1149205545=5, srv232761688=65, srv1633005542=40, srv756924040=90, srv618135364=80, srv619863180=81, srv271126238=68, srv1167876626=7, srv1422477052=25, srv1428061628=26, srv2040636407=59, srv756521473=89, srv813394698=94, srv1314295973=19, srv597916296=79, srv1556640497=35} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:53:13,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:53:13,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:53:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:53:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:53:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:53:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:53:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:53:13,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:53:13,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:53:13,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:53:13,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-17 20:53:13,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-17 20:53:13,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-17 20:53:13,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-17 20:53:13,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-17 20:53:13,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-17 20:53:13,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-17 20:53:13,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-17 20:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-17 20:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-17 20:53:13,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-17 20:53:13,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04219149451905964, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054042008849955976, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04219149451905964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16925, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23758333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:13,852 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-17 20:53:13,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1742073227340134, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04219149451905964, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.054042008849955976, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04219149451905964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16925, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23758333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-17 20:53:14,255 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 559 ms to try 5000 different iterations.  Found a solution that moves 2621 regions; Going from a computed imbalance of 0.1742073227340134 to a new imbalance of 0.014814669419505091. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022533177932642073); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031283792055029264, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2912222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022533177932642073); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:14,270 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1249551768:77 , srv2038673266:78 , srv1183514945:79 , srv1234884198:80 , srv2122350848:80 , srv1149205545:81 , srv2104565986:81 , srv620526058:81 , srv813394698:81 , srv937442501:81 , srv1617097576:82 , srv1633005542:82 , srv2032385448:82 , srv756924040:82 , srv1084845221:83 , srv1135458264:83 , srv1278253750:83 , srv1464223346:83 , srv1472376905:83 , srv2048685802:83 , srv618135364:83 , srv685104233:83 , srv691989676:83 , srv1337960120:84 , srv1551974262:84 , srv1774390419:84 , srv1796235398:84 , srv1930389005:84 , srv255998003:84 , srv301312584:84 , srv303416331:84 , srv788310827:84 , srv1044814675:85 , srv1167876626:85 , srv229843757:85 , srv271126238:85 , srv276830591:85 , srv430398515:85 , srv66371267:85 , srv743975458:85 , srv956085974:85 , srv1323486188:86 , srv1343333727:86 , srv147040010:86 , srv1516049960:86 , srv1529760856:86 , srv1571281630:86 , srv1631375250:86 , srv1824889112:86 , srv1997742674:86 , srv199946131:86 , srv232761688:86 , srv1132720962:87 , srv1242811274:87 , srv1253726843:87 , srv1273742411:87 , srv1380960150:87 , srv1533365032:87 , srv1585257851:87 , srv1639967867:87 , srv1779015521:87 , srv1848348786:87 , srv1945090755:87 , srv2040636407:87 , srv276688374:87 , srv362647944:87 , srv506446051:87 , srv631266051:87 , srv756521473:87 , srv773229433:87 , srv903225095:87 , srv1269257415:88 , srv1303131988:88 , srv1314295973:88 , srv1353871419:88 , srv1428061628:88 , srv1665070281:88 , srv1729376307:88 , srv230751483:88 , srv259780349:88 , srv467827834:88 , srv547158772:88 , srv671983390:88 , srv797905465:88 , srv1422477052:89 , srv1556640497:89 , srv619863180:89 , srv1149472337:90 , srv1872170029:90 , srv1976279153:90 , srv2019813955:90 , srv457415407:90 , srv597916296:90 , srv912742670:90 , srv1216238641:91 , srv1463005023:91 , srv1963923118:91 , srv915317931:91 , srv1133814941:92 , srv1310810894:509 }
2023-07-17 20:53:14,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:53:14,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv691989676=87, srv301312584=71, srv303416331=72, srv1044814675=0, srv199946131=55, srv1639967867=41, srv1872170029=49, srv547158772=78, srv457415407=75, srv147040010=29, srv1463005023=27, srv259780349=67, srv230751483=64, srv937442501=98, srv915317931=97, srv1084845221=1, srv1343333727=22, srv797905465=93, srv1585257851=37, srv2032385448=57, srv276830591=70, srv1242811274=11, srv631266051=83, srv1132720962=2, srv2048685802=60, srv1617097576=38, srv671983390=85, srv1551974262=34, srv1631375250=39, srv1779015521=45, srv1249551768=12, srv912742670=96, srv773229433=91, srv2122350848=62, srv788310827=92, srv1976279153=53, srv1253726843=13, srv685104233=86, srv1997742674=54, srv1133814941=3, srv2104565986=61, srv255998003=66, srv1310810894=18, srv1234884198=10, srv1464223346=28, srv1848348786=48, srv1774390419=44, srv620526058=82, srv1323486188=20, srv66371267=84, srv956085974=99, srv1183514945=8, srv1380960150=24, srv1135458264=4, srv1945090755=51, srv1273742411=15, srv1529760856=32, srv1796235398=46, srv430398515=74, srv1571281630=36, srv1533365032=33, srv2038673266=58, srv276688374=69, srv1472376905=30, srv1269257415=14, srv229843757=63, srv1516049960=31, srv1353871419=23, srv1824889112=47, srv1729376307=43, srv1216238641=9, srv1303131988=17, srv362647944=73, srv1665070281=42, srv743975458=88, srv1930389005=50, srv2019813955=56, srv1278253750=16, srv1963923118=52, srv506446051=77, srv467827834=76, srv1149472337=6, srv1337960120=21, srv903225095=95, srv1149205545=5, srv232761688=65, srv1633005542=40, srv756924040=90, srv618135364=80, srv619863180=81, srv271126238=68, srv1167876626=7, srv1422477052=25, srv1428061628=26, srv2040636407=59, srv756521473=89, srv813394698=94, srv1314295973=19, srv597916296=79, srv1556640497=35} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:53:14,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:53:14,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:53:14,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:53:14,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:53:14,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:53:14,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:53:14,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:53:14,301 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-17 20:53:14,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-17 20:53:14,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-17 20:53:14,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-17 20:53:14,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-17 20:53:14,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-17 20:53:14,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-17 20:53:14,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-17 20:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-17 20:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-17 20:53:14,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-17 20:53:14,465 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022533177932642073); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031283792055029264, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022533177932642073); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:14,465 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-17 20:53:14,466 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014796316785246373, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022533177932642073); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.031283792055029264, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022533177932642073); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.011583333333333333); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04575, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-17 20:53:14,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 507 ms to try 5000 different iterations.  Found a solution that moves 1017 regions; Going from a computed imbalance of 0.014796316785246373 to a new imbalance of 8.711592244031752E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005781057516857846); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); MoveCostFunction : (multiplier=7.0, imbalance=0.113, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005781057516857846); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:14,856 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv467827834:80 , srv1463005023:81 , srv1343333727:82 , srv1631375250:82 , srv915317931:82 , srv1044814675:83 , srv1132720962:83 , srv1133814941:83 , srv1149472337:83 , srv1571281630:83 , srv1665070281:83 , srv1976279153:83 , srv2019813955:83 , srv271126238:83 , srv276688374:83 , srv1216238641:84 , srv1234884198:84 , srv1796235398:84 , srv1963923118:84 , srv230751483:84 , srv457415407:84 , srv631266051:84 , srv671983390:84 , srv813394698:84 , srv903225095:84 , srv1242811274:85 , srv1253726843:85 , srv1269257415:85 , srv1314295973:85 , srv1337960120:85 , srv1422477052:85 , srv1428061628:85 , srv1464223346:85 , srv147040010:85 , srv1472376905:85 , srv1551974262:85 , srv1556640497:85 , srv1585257851:85 , srv1824889112:85 , srv1848348786:85 , srv1872170029:85 , srv1945090755:85 , srv2040636407:85 , srv255998003:85 , srv362647944:85 , srv66371267:85 , srv756924040:85 , srv1149205545:86 , srv1303131988:86 , srv1323486188:86 , srv1353871419:86 , srv1516049960:86 , srv1529760856:86 , srv1533365032:86 , srv1617097576:86 , srv1639967867:86 , srv1729376307:86 , srv1779015521:86 , srv1997742674:86 , srv2048685802:86 , srv2122350848:86 , srv229843757:86 , srv619863180:86 , srv691989676:86 , srv743975458:86 , srv756521473:86 , srv773229433:86 , srv956085974:86 , srv1183514945:87 , srv1278253750:87 , srv1380960150:87 , srv1774390419:87 , srv1930389005:87 , srv199946131:87 , srv2032385448:87 , srv2038673266:87 , srv2104565986:87 , srv259780349:87 , srv276830591:87 , srv301312584:87 , srv303416331:87 , srv506446051:87 , srv547158772:87 , srv597916296:87 , srv620526058:87 , srv797905465:87 , srv912742670:87 , srv937442501:87 , srv1084845221:88 , srv1167876626:88 , srv1633005542:88 , srv232761688:88 , srv430398515:88 , srv618135364:88 , srv685104233:88 , srv788310827:88 , srv1135458264:89 , srv1249551768:89 , srv1273742411:90 , srv1310810894:535 }
2023-07-17 20:53:14,863 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:53:14,864 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv691989676=87, srv301312584=71, srv303416331=72, srv1044814675=0, srv199946131=55, srv1639967867=41, srv1872170029=49, srv547158772=78, srv457415407=75, srv147040010=29, srv1463005023=27, srv259780349=67, srv230751483=64, srv937442501=98, srv915317931=97, srv1084845221=1, srv1343333727=22, srv797905465=93, srv1585257851=37, srv2032385448=57, srv276830591=70, srv1242811274=11, srv631266051=83, srv1132720962=2, srv2048685802=60, srv1617097576=38, srv671983390=85, srv1551974262=34, srv1631375250=39, srv1779015521=45, srv1249551768=12, srv912742670=96, srv773229433=91, srv2122350848=62, srv788310827=92, srv1976279153=53, srv1253726843=13, srv685104233=86, srv1997742674=54, srv1133814941=3, srv2104565986=61, srv255998003=66, srv1310810894=18, srv1234884198=10, srv1464223346=28, srv1848348786=48, srv1774390419=44, srv620526058=82, srv1323486188=20, srv66371267=84, srv956085974=99, srv1183514945=8, srv1380960150=24, srv1135458264=4, srv1945090755=51, srv1273742411=15, srv1529760856=32, srv1796235398=46, srv430398515=74, srv1571281630=36, srv1533365032=33, srv2038673266=58, srv276688374=69, srv1472376905=30, srv1269257415=14, srv229843757=63, srv1516049960=31, srv1353871419=23, srv1824889112=47, srv1729376307=43, srv1216238641=9, srv1303131988=17, srv362647944=73, srv1665070281=42, srv743975458=88, srv1930389005=50, srv2019813955=56, srv1278253750=16, srv1963923118=52, srv506446051=77, srv467827834=76, srv1149472337=6, srv1337960120=21, srv903225095=95, srv1149205545=5, srv232761688=65, srv1633005542=40, srv756924040=90, srv618135364=80, srv619863180=81, srv271126238=68, srv1167876626=7, srv1422477052=25, srv1428061628=26, srv2040636407=59, srv756521473=89, srv813394698=94, srv1314295973=19, srv597916296=79, srv1556640497=35} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:53:14,889 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:53:14,890 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:53:14,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:53:14,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-17 20:53:14,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-17 20:53:14,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-17 20:53:14,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-17 20:53:14,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-17 20:53:14,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-17 20:53:14,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-17 20:53:14,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-17 20:53:14,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-17 20:53:14,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-17 20:53:15,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005781057516857846); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005781057516857846); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:15,015 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-07-17 20:53:15,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.640380381990105E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005781057516857846); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005781057516857846); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-07-17 20:53:15,320 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 405 ms to try 5000 different iterations.  Found a solution that moves 261 regions; Going from a computed imbalance of 8.640380381990105E-4 to a new imbalance of 2.1420954314611457E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.029, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:15,321 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv912742670:82 , srv1273742411:83 , srv1963923118:83 , srv2104565986:83 , srv259780349:83 , srv620526058:83 , srv1044814675:84 , srv1132720962:84 , srv1135458264:84 , srv1149205545:84 , srv1149472337:84 , srv1167876626:84 , srv1183514945:84 , srv1249551768:84 , srv1269257415:84 , srv1303131988:84 , srv1323486188:84 , srv1343333727:84 , srv1422477052:84 , srv1428061628:84 , srv1463005023:84 , srv1464223346:84 , srv1529760856:84 , srv1617097576:84 , srv1633005542:84 , srv1729376307:84 , srv1779015521:84 , srv1824889112:84 , srv1872170029:84 , srv1930389005:84 , srv1976279153:84 , srv2032385448:84 , srv271126238:84 , srv362647944:84 , srv430398515:84 , srv457415407:84 , srv547158772:84 , srv619863180:84 , srv631266051:84 , srv685104233:84 , srv691989676:84 , srv743975458:84 , srv797905465:84 , srv915317931:84 , srv937442501:84 , srv1084845221:85 , srv1133814941:85 , srv1216238641:85 , srv1234884198:85 , srv1242811274:85 , srv1253726843:85 , srv1278253750:85 , srv1314295973:85 , srv1353871419:85 , srv1380960150:85 , srv147040010:85 , srv1472376905:85 , srv1516049960:85 , srv1533365032:85 , srv1551974262:85 , srv1556640497:85 , srv1571281630:85 , srv1585257851:85 , srv1631375250:85 , srv1639967867:85 , srv1665070281:85 , srv1774390419:85 , srv1796235398:85 , srv1848348786:85 , srv1945090755:85 , srv1997742674:85 , srv2019813955:85 , srv2038673266:85 , srv2040636407:85 , srv2048685802:85 , srv2122350848:85 , srv229843757:85 , srv230751483:85 , srv232761688:85 , srv255998003:85 , srv276688374:85 , srv276830591:85 , srv301312584:85 , srv303416331:85 , srv467827834:85 , srv506446051:85 , srv597916296:85 , srv618135364:85 , srv66371267:85 , srv756521473:85 , srv756924040:85 , srv773229433:85 , srv788310827:85 , srv903225095:85 , srv956085974:85 , srv1337960120:86 , srv199946131:86 , srv671983390:86 , srv813394698:86 , srv1310810894:633 }
2023-07-17 20:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:53:15,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv691989676=87, srv301312584=71, srv303416331=72, srv1044814675=0, srv199946131=55, srv1639967867=41, srv1872170029=49, srv547158772=78, srv457415407=75, srv147040010=29, srv1463005023=27, srv259780349=67, srv230751483=64, srv937442501=98, srv915317931=97, srv1084845221=1, srv1343333727=22, srv797905465=93, srv1585257851=37, srv2032385448=57, srv276830591=70, srv1242811274=11, srv631266051=83, srv1132720962=2, srv2048685802=60, srv1617097576=38, srv671983390=85, srv1551974262=34, srv1631375250=39, srv1779015521=45, srv1249551768=12, srv912742670=96, srv773229433=91, srv2122350848=62, srv788310827=92, srv1976279153=53, srv1253726843=13, srv685104233=86, srv1997742674=54, srv1133814941=3, srv2104565986=61, srv255998003=66, srv1310810894=18, srv1234884198=10, srv1464223346=28, srv1848348786=48, srv1774390419=44, srv620526058=82, srv1323486188=20, srv66371267=84, srv956085974=99, srv1183514945=8, srv1380960150=24, srv1135458264=4, srv1945090755=51, srv1273742411=15, srv1529760856=32, srv1796235398=46, srv430398515=74, srv1571281630=36, srv1533365032=33, srv2038673266=58, srv276688374=69, srv1472376905=30, srv1269257415=14, srv229843757=63, srv1516049960=31, srv1353871419=23, srv1824889112=47, srv1729376307=43, srv1216238641=9, srv1303131988=17, srv362647944=73, srv1665070281=42, srv743975458=88, srv1930389005=50, srv2019813955=56, srv1278253750=16, srv1963923118=52, srv506446051=77, srv467827834=76, srv1149472337=6, srv1337960120=21, srv903225095=95, srv1149205545=5, srv232761688=65, srv1633005542=40, srv756924040=90, srv618135364=80, srv619863180=81, srv271126238=68, srv1167876626=7, srv1422477052=25, srv1428061628=26, srv2040636407=59, srv756521473=89, srv813394698=94, srv1314295973=19, srv597916296=79, srv1556640497=35} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:53:15,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-07-17 20:53:15,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-07-17 20:53:15,338 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-07-17 20:53:15,339 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-07-17 20:53:15,340 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-07-17 20:53:15,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-07-17 20:53:15,456 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.959339325336565E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.868392354571063E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.868392354571063E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:15,457 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv912742670:82 , srv1273742411:83 , srv1963923118:83 , srv2104565986:83 , srv259780349:83 , srv620526058:83 , srv1044814675:84 , srv1132720962:84 , srv1135458264:84 , srv1149205545:84 , srv1149472337:84 , srv1167876626:84 , srv1183514945:84 , srv1249551768:84 , srv1269257415:84 , srv1303131988:84 , srv1323486188:84 , srv1343333727:84 , srv1422477052:84 , srv1428061628:84 , srv1463005023:84 , srv1464223346:84 , srv1529760856:84 , srv1617097576:84 , srv1633005542:84 , srv1729376307:84 , srv1779015521:84 , srv1824889112:84 , srv1872170029:84 , srv1930389005:84 , srv1976279153:84 , srv2032385448:84 , srv271126238:84 , srv362647944:84 , srv430398515:84 , srv457415407:84 , srv547158772:84 , srv619863180:84 , srv631266051:84 , srv685104233:84 , srv691989676:84 , srv743975458:84 , srv797905465:84 , srv915317931:84 , srv937442501:84 , srv1084845221:85 , srv1133814941:85 , srv1216238641:85 , srv1234884198:85 , srv1242811274:85 , srv1253726843:85 , srv1278253750:85 , srv1314295973:85 , srv1353871419:85 , srv1380960150:85 , srv147040010:85 , srv1472376905:85 , srv1516049960:85 , srv1533365032:85 , srv1551974262:85 , srv1556640497:85 , srv1571281630:85 , srv1585257851:85 , srv1631375250:85 , srv1639967867:85 , srv1665070281:85 , srv1774390419:85 , srv1796235398:85 , srv1848348786:85 , srv1945090755:85 , srv1997742674:85 , srv2019813955:85 , srv2038673266:85 , srv2040636407:85 , srv2048685802:85 , srv2122350848:85 , srv229843757:85 , srv230751483:85 , srv232761688:85 , srv255998003:85 , srv276688374:85 , srv276830591:85 , srv301312584:85 , srv303416331:85 , srv467827834:85 , srv506446051:85 , srv597916296:85 , srv618135364:85 , srv66371267:85 , srv756521473:85 , srv756924040:85 , srv773229433:85 , srv788310827:85 , srv903225095:85 , srv956085974:85 , srv1337960120:86 , srv199946131:86 , srv671983390:86 , srv813394698:86 , srv1310810894:633 }
2023-07-17 20:53:15,488 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=357 (was 301) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=7017 (was 7614)
2023-07-17 20:53:15,499 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=357, ProcessCount=172, AvailableMemoryMB=7016
2023-07-17 20:53:15,500 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-07-17 20:53:15,501 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-07-17 20:53:15,515 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1000208101:84 , srv1041443534:84 , srv1116066093:84 , srv1153167418:84 , srv115796579:84 , srv1159653267:84 , srv1224760707:84 , srv1258899803:84 , srv135689029:84 , srv1482716174:84 , srv1666752824:84 , srv1681311800:84 , srv1695142113:84 , srv1699742942:84 , srv1740017877:84 , srv1869216280:84 , srv1965960463:84 , srv2092886722:84 , srv215254031:84 , srv256911624:84 , srv261692016:84 , srv396009740:84 , srv477442081:84 , srv550725959:84 , srv63542780:84 , srv758757172:84 , srv841889831:84 , srv908947969:84 , srv937169305:84 , srv462313975:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-07-17 20:53:15,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:53:15,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1695142113=12, srv1258899803=7, srv1159653267=5, srv1699742942=13, srv1666752824=10, srv1041443534=1, srv462313975=22, srv1965960463=16, srv1681311800=11, srv477442081=23, srv1740017877=14, srv758757172=26, srv1869216280=15, srv841889831=27, srv908947969=28, srv256911624=19, srv1153167418=3, srv2092886722=17, srv215254031=18, srv63542780=25, srv135689029=8, srv396009740=21, srv1224760707=6, srv1482716174=9, srv261692016=20, srv115796579=4, srv937169305=29, srv1000208101=0, srv1116066093=2, srv550725959=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:53:15,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:53:15,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:53:15,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:53:15,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:53:15,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-17 20:53:15,558 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0755739193156726, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:53:15,558 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912431174216328, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0755739193156726, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-07-17 20:55:15,536 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5691065 different iterations.  Found a solution that moves 1863 regions; Going from a computed imbalance of 0.9912431174216328 to a new imbalance of 4.3483349388262195E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.69, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:55:15,542 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1000208101:90 , srv1041443534:90 , srv1116066093:90 , srv1153167418:90 , srv115796579:90 , srv1159653267:90 , srv1224760707:90 , srv1258899803:90 , srv135689029:90 , srv1482716174:90 , srv1666752824:90 , srv1681311800:90 , srv1695142113:90 , srv1699742942:90 , srv1740017877:90 , srv1869216280:90 , srv1965960463:90 , srv2092886722:90 , srv215254031:90 , srv256911624:90 , srv261692016:90 , srv396009740:90 , srv462313975:90 , srv477442081:90 , srv550725959:90 , srv63542780:90 , srv758757172:90 , srv841889831:90 , srv908947969:90 , srv937169305:90 }
2023-07-17 20:55:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-07-17 20:55:15,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1695142113=12, srv1258899803=7, srv1159653267=5, srv1699742942=13, srv1666752824=10, srv1041443534=1, srv462313975=22, srv1965960463=16, srv1681311800=11, srv477442081=23, srv1740017877=14, srv758757172=26, srv1869216280=15, srv841889831=27, srv908947969=28, srv256911624=19, srv1153167418=3, srv2092886722=17, srv215254031=18, srv63542780=25, srv135689029=8, srv396009740=21, srv1224760707=6, srv1482716174=9, srv261692016=20, srv115796579=4, srv937169305=29, srv1000208101=0, srv1116066093=2, srv550725959=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-07-17 20:55:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-07-17 20:55:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-07-17 20:55:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-07-17 20:55:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-07-17 20:55:15,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-07-17 20:55:15,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-07-17 20:55:15,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-07-17 20:55:15,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-07-17 20:55:15,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-07-17 20:55:15,565 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-07-17 20:55:15,565 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1000208101:90 , srv1041443534:90 , srv1116066093:90 , srv1153167418:90 , srv115796579:90 , srv1159653267:90 , srv1224760707:90 , srv1258899803:90 , srv135689029:90 , srv1482716174:90 , srv1666752824:90 , srv1681311800:90 , srv1695142113:90 , srv1699742942:90 , srv1740017877:90 , srv1869216280:90 , srv1965960463:90 , srv2092886722:90 , srv215254031:90 , srv256911624:90 , srv261692016:90 , srv396009740:90 , srv462313975:90 , srv477442081:90 , srv550725959:90 , srv63542780:90 , srv758757172:90 , srv841889831:90 , srv908947969:90 , srv937169305:90 }
2023-07-17 20:55:15,587 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 357), ProcessCount=172 (was 172), AvailableMemoryMB=6516 (was 7016)