2024-03-09 12:53:19,311 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2024-03-09 12:53:19,633 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2024-03-09 12:53:19,745 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2024-03-09 12:53:19,807 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-09 12:53:19,815 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-09 12:53:19,830 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=171, AvailableMemoryMB=11772
2024-03-09 12:53:19,832 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-09 12:53:19,833 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2024-03-09 12:53:19,970 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1066358423:84 , srv1069413400:84 , srv111433994:84 , srv1122854056:84 , srv1129277650:84 , srv1152381912:84 , srv1166626457:84 , srv1167585621:84 , srv1173987201:84 , srv1197215321:84 , srv1234966172:84 , srv1252876366:84 , srv1257027836:84 , srv1257926327:84 , srv1260944913:84 , srv1279546861:84 , srv1308220682:84 , srv1335142388:84 , srv1389499720:84 , srv1408472203:84 , srv1409497334:84 , srv1424744835:84 , srv1452502368:84 , srv1458739833:84 , srv1469652102:84 , srv1512898504:84 , srv1519458479:84 , srv1523688389:84 , srv1533058838:84 , srv1534555459:84 , srv1544209436:84 , srv1554645492:84 , srv1555514162:84 , srv1566315412:84 , srv15895231:84 , srv1644986708:84 , srv1652932689:84 , srv1688095868:84 , srv1688755249:84 , srv1689173515:84 , srv1746673477:84 , srv1787907478:84 , srv1862676486:84 , srv1891001976:84 , srv1897455423:84 , srv1915580374:84 , srv1916016982:84 , srv1917669639:84 , srv1967016495:84 , srv1987881524:84 , srv1994833678:84 , srv2002506487:84 , srv2005036943:84 , srv202620503:84 , srv2077960635:84 , srv2084589250:84 , srv2086385952:84 , srv2089267079:84 , srv2096836325:84 , srv2116079752:84 , srv212607282:84 , srv219355945:84 , srv223154906:84 , srv239465894:84 , srv245272677:84 , srv245861660:84 , srv269956129:84 , srv287798527:84 , srv291971214:84 , srv304243223:84 , srv335579562:84 , srv340745019:84 , srv389970669:84 , srv413801258:84 , srv424342086:84 , srv454853338:84 , srv506724283:84 , srv511595041:84 , srv53282203:84 , srv557106702:84 , srv597350063:84 , srv63837722:84 , srv640651210:84 , srv693624221:84 , srv757989544:84 , srv785713128:84 , srv800176387:84 , srv84232548:84 , srv855232642:84 , srv857599436:84 , srv860851164:84 , srv870121093:84 , srv880309823:84 , srv891368953:84 , srv904107282:84 , srv931322854:84 , srv948688173:84 , srv963845472:84 , srv968475248:84 , srv227393303:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2024-03-09 12:53:19,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:53:19,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167585621=7, srv860851164=91, srv1566315412=33, srv1452502368=22, srv2089267079=57, srv1279546861=15, srv1523688389=27, srv287798527=68, srv1746673477=40, srv2116079752=59, srv269956129=67, srv340745019=72, srv640651210=83, srv1257027836=12, srv1260944913=14, srv291971214=69, srv304243223=70, srv202620503=53, srv1688095868=37, srv1166626457=6, srv870121093=92, srv1409497334=20, srv1152381912=5, srv1994833678=50, srv1652932689=36, srv1891001976=43, srv880309823=93, srv968475248=99, srv2084589250=55, srv1122854056=3, srv1688755249=38, srv335579562=71, srv239465894=64, srv1916016982=46, srv963845472=98, srv1458739833=23, srv1197215321=9, srv800176387=87, srv223154906=62, srv63837722=82, srv111433994=2, srv693624221=84, srv2077960635=54, srv424342086=75, srv53282203=79, srv1554645492=31, srv1967016495=48, srv1408472203=19, srv245272677=65, srv557106702=80, srv1069413400=1, srv1917669639=47, srv84232548=88, srv1512898504=25, srv219355945=61, srv1555514162=32, srv2086385952=56, srv597350063=81, srv785713128=86, srv212607282=60, srv245861660=66, srv413801258=74, srv857599436=90, srv2002506487=51, srv1987881524=49, srv1234966172=10, srv1534555459=29, srv1862676486=42, srv2096836325=58, srv1424744835=21, srv511595041=78, srv855232642=89, srv891368953=94, srv757989544=85, srv904107282=95, srv1915580374=45, srv389970669=73, srv931322854=96, srv1066358423=0, srv1257926327=13, srv1173987201=8, srv1644986708=35, srv948688173=97, srv1519458479=26, srv1469652102=24, srv15895231=34, srv506724283=77, srv1897455423=44, srv1252876366=11, srv454853338=76, srv1533058838=28, srv1129277650=4, srv1689173515=39, srv1389499720=18, srv1787907478=41, srv1308220682=16, srv1335142388=17, srv1544209436=30, srv227393303=63, srv2005036943=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:53:20,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:53:20,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:53:20,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:53:20,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:53:20,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:53:20,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:53:20,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:53:20,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:53:20,103 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:53:20,104 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:53:20,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:53:20,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:53:20,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:53:20,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:53:20,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:53:20,105 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-09 12:53:20,106 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-09 12:53:20,107 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-09 12:53:20,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-09 12:53:20,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-09 12:53:20,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-09 12:53:20,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-09 12:53:20,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-09 12:53:20,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-09 12:53:20,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-09 12:53:20,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-09 12:53:20,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-09 12:53:20,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-09 12:53:20,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-09 12:53:20,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-09 12:53:20,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-09 12:53:20,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-09 12:53:20,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-09 12:53:20,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-09 12:53:20,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-09 12:53:20,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-09 12:53:20,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-09 12:53:20,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-09 12:53:20,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-09 12:53:20,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-09 12:53:20,563 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:20,564 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-09 12:53:20,564 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-09 12:53:21,425 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 1277 ms to try 5000 different iterations.  Found a solution that moves 4641 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18665858131676188. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03928053549643124, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05603430086274126, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5156666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03928053549643124, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18258333333333332, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:21,443 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1066358423:80 , srv15895231:81 , srv1152381912:82 , srv1257926327:82 , srv219355945:82 , srv506724283:82 , srv640651210:82 , srv1408472203:83 , srv1469652102:83 , srv1519458479:83 , srv1566315412:83 , srv1862676486:83 , srv1917669639:83 , srv800176387:83 , srv870121093:83 , srv948688173:83 , srv2084589250:84 , srv304243223:84 , srv340745019:84 , srv931322854:84 , srv1167585621:85 , srv1554645492:85 , srv1555514162:85 , srv1688095868:85 , srv1689173515:85 , srv1746673477:85 , srv1787907478:85 , srv1897455423:85 , srv1994833678:85 , srv2002506487:85 , srv2096836325:85 , srv2116079752:85 , srv291971214:85 , srv63837722:85 , srv757989544:85 , srv963845472:85 , srv1069413400:86 , srv111433994:86 , srv1129277650:86 , srv1197215321:86 , srv1252876366:86 , srv1534555459:86 , srv1987881524:86 , srv2086385952:86 , srv212607282:86 , srv239465894:86 , srv245861660:86 , srv269956129:86 , srv557106702:86 , srv693624221:86 , srv84232548:86 , srv880309823:86 , srv1173987201:87 , srv1234966172:87 , srv1335142388:87 , srv1409497334:87 , srv1424744835:87 , srv1891001976:87 , srv2089267079:87 , srv223154906:87 , srv287798527:87 , srv335579562:87 , srv860851164:87 , srv904107282:87 , srv1122854056:88 , srv1279546861:88 , srv1512898504:88 , srv1523688389:88 , srv1533058838:88 , srv1652932689:88 , srv1688755249:88 , srv2005036943:88 , srv202620503:88 , srv2077960635:88 , srv245272677:88 , srv413801258:88 , srv857599436:88 , srv968475248:88 , srv1544209436:89 , srv1916016982:89 , srv424342086:89 , srv855232642:89 , srv1166626457:90 , srv1257027836:90 , srv1260944913:90 , srv1308220682:90 , srv1389499720:90 , srv1452502368:90 , srv1915580374:90 , srv1967016495:90 , srv454853338:90 , srv511595041:90 , srv597350063:90 , srv891368953:90 , srv1644986708:91 , srv389970669:91 , srv53282203:92 , srv785713128:92 , srv1458739833:93 , srv227393303:439 }
2024-03-09 12:53:21,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:53:21,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167585621=7, srv860851164=91, srv1566315412=33, srv1452502368=22, srv2089267079=57, srv1279546861=15, srv1523688389=27, srv287798527=68, srv1746673477=40, srv2116079752=59, srv269956129=67, srv340745019=72, srv640651210=83, srv1257027836=12, srv1260944913=14, srv291971214=69, srv304243223=70, srv202620503=53, srv1688095868=37, srv1166626457=6, srv870121093=92, srv1409497334=20, srv1152381912=5, srv1994833678=50, srv1652932689=36, srv1891001976=43, srv880309823=93, srv968475248=99, srv2084589250=55, srv1122854056=3, srv1688755249=38, srv335579562=71, srv239465894=64, srv1916016982=46, srv963845472=98, srv1458739833=23, srv1197215321=9, srv800176387=87, srv223154906=62, srv63837722=82, srv111433994=2, srv693624221=84, srv2077960635=54, srv424342086=75, srv53282203=79, srv1554645492=31, srv1967016495=48, srv1408472203=19, srv245272677=65, srv557106702=80, srv1069413400=1, srv1917669639=47, srv84232548=88, srv1512898504=25, srv219355945=61, srv1555514162=32, srv2086385952=56, srv597350063=81, srv785713128=86, srv212607282=60, srv245861660=66, srv413801258=74, srv857599436=90, srv2002506487=51, srv1987881524=49, srv1234966172=10, srv1534555459=29, srv1862676486=42, srv2096836325=58, srv1424744835=21, srv511595041=78, srv855232642=89, srv891368953=94, srv757989544=85, srv904107282=95, srv1915580374=45, srv389970669=73, srv931322854=96, srv1066358423=0, srv1257926327=13, srv1173987201=8, srv1644986708=35, srv948688173=97, srv1519458479=26, srv1469652102=24, srv15895231=34, srv506724283=77, srv1897455423=44, srv1252876366=11, srv454853338=76, srv1533058838=28, srv1129277650=4, srv1689173515=39, srv1389499720=18, srv1787907478=41, srv1308220682=16, srv1335142388=17, srv1544209436=30, srv227393303=63, srv2005036943=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:53:21,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:53:21,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:53:21,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:53:21,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:53:21,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:53:21,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:53:21,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:53:21,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:53:21,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:53:21,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:53:21,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:53:21,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:53:21,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:53:21,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:53:21,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:53:21,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:53:21,485 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:53:21,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:53:21,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:53:21,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:53:21,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:53:21,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:53:21,486 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:53:21,487 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-09 12:53:21,488 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-09 12:53:21,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-09 12:53:21,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-09 12:53:21,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-09 12:53:21,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-09 12:53:21,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-09 12:53:21,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-09 12:53:21,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-09 12:53:21,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-09 12:53:21,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-09 12:53:21,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-09 12:53:21,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-09 12:53:21,492 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-09 12:53:21,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-09 12:53:21,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-09 12:53:21,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-09 12:53:21,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-09 12:53:21,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-09 12:53:21,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-09 12:53:21,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-09 12:53:21,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-09 12:53:21,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-09 12:53:21,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-09 12:53:21,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-09 12:53:21,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-09 12:53:21,684 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03928053549643124, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05603430086274126, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03928053549643124, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18258333333333332, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:21,685 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-09 12:53:21,685 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18662608434019007, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03928053549643124, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05603430086274126, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03928053549643124, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18258333333333332, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24225, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-09 12:53:21,993 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 455 ms to try 5000 different iterations.  Found a solution that moves 2653 regions; Going from a computed imbalance of 0.18662608434019007 to a new imbalance of 0.013695924320667551. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02104869975951674); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03262204056867377, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2947777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02104869975951674); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:22,007 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1458739833:79 , srv597350063:80 , srv785713128:80 , srv1257027836:81 , srv1260944913:81 , srv1452502368:81 , srv389970669:81 , srv454853338:81 , srv511595041:81 , srv855232642:81 , srv1166626457:82 , srv1234966172:83 , srv1279546861:83 , srv1308220682:83 , srv1335142388:83 , srv1523688389:83 , srv1533058838:83 , srv1891001976:83 , srv1915580374:83 , srv1967016495:83 , srv2077960635:83 , srv2089267079:83 , srv424342086:83 , srv1069413400:84 , srv1197215321:84 , srv1644986708:84 , srv2005036943:84 , srv212607282:84 , srv287798527:84 , srv413801258:84 , srv53282203:84 , srv693624221:84 , srv84232548:84 , srv860851164:84 , srv891368953:84 , srv1173987201:85 , srv1252876366:85 , srv1512898504:85 , srv1544209436:85 , srv1652932689:85 , srv1916016982:85 , srv2086385952:85 , srv239465894:85 , srv245861660:85 , srv269956129:85 , srv291971214:85 , srv904107282:85 , srv968475248:85 , srv1389499720:86 , srv1424744835:86 , srv1688755249:86 , srv1689173515:86 , srv1897455423:86 , srv1987881524:86 , srv2002506487:86 , srv202620503:86 , srv2116079752:86 , srv245272677:86 , srv335579562:86 , srv557106702:86 , srv63837722:86 , srv857599436:86 , srv111433994:87 , srv1122854056:87 , srv1129277650:87 , srv1554645492:87 , srv1994833678:87 , srv2096836325:87 , srv223154906:87 , srv880309823:87 , srv963845472:87 , srv1409497334:88 , srv1534555459:88 , srv1688095868:88 , srv1746673477:88 , srv1787907478:88 , srv1862676486:88 , srv2084589250:88 , srv304243223:88 , srv340745019:88 , srv948688173:88 , srv1408472203:89 , srv1469652102:89 , srv1519458479:89 , srv1555514162:89 , srv15895231:89 , srv1917669639:89 , srv757989544:89 , srv800176387:89 , srv870121093:89 , srv931322854:89 , srv1152381912:90 , srv1167585621:90 , srv1257926327:90 , srv1566315412:90 , srv506724283:90 , srv640651210:90 , srv1066358423:92 , srv219355945:92 , srv227393303:522 }
2024-03-09 12:53:22,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:53:22,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167585621=7, srv860851164=91, srv1566315412=33, srv1452502368=22, srv2089267079=57, srv1279546861=15, srv1523688389=27, srv287798527=68, srv1746673477=40, srv2116079752=59, srv269956129=67, srv340745019=72, srv640651210=83, srv1257027836=12, srv1260944913=14, srv291971214=69, srv304243223=70, srv202620503=53, srv1688095868=37, srv1166626457=6, srv870121093=92, srv1409497334=20, srv1152381912=5, srv1994833678=50, srv1652932689=36, srv1891001976=43, srv880309823=93, srv968475248=99, srv2084589250=55, srv1122854056=3, srv1688755249=38, srv335579562=71, srv239465894=64, srv1916016982=46, srv963845472=98, srv1458739833=23, srv1197215321=9, srv800176387=87, srv223154906=62, srv63837722=82, srv111433994=2, srv693624221=84, srv2077960635=54, srv424342086=75, srv53282203=79, srv1554645492=31, srv1967016495=48, srv1408472203=19, srv245272677=65, srv557106702=80, srv1069413400=1, srv1917669639=47, srv84232548=88, srv1512898504=25, srv219355945=61, srv1555514162=32, srv2086385952=56, srv597350063=81, srv785713128=86, srv212607282=60, srv245861660=66, srv413801258=74, srv857599436=90, srv2002506487=51, srv1987881524=49, srv1234966172=10, srv1534555459=29, srv1862676486=42, srv2096836325=58, srv1424744835=21, srv511595041=78, srv855232642=89, srv891368953=94, srv757989544=85, srv904107282=95, srv1915580374=45, srv389970669=73, srv931322854=96, srv1066358423=0, srv1257926327=13, srv1173987201=8, srv1644986708=35, srv948688173=97, srv1519458479=26, srv1469652102=24, srv15895231=34, srv506724283=77, srv1897455423=44, srv1252876366=11, srv454853338=76, srv1533058838=28, srv1129277650=4, srv1689173515=39, srv1389499720=18, srv1787907478=41, srv1308220682=16, srv1335142388=17, srv1544209436=30, srv227393303=63, srv2005036943=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:53:22,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:53:22,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:53:22,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:53:22,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:53:22,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:53:22,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-09 12:53:22,040 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-09 12:53:22,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-09 12:53:22,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-09 12:53:22,041 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-09 12:53:22,042 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-09 12:53:22,043 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-09 12:53:22,044 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-09 12:53:22,045 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-09 12:53:22,046 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-09 12:53:22,047 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-09 12:53:22,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-09 12:53:22,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-09 12:53:22,048 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-09 12:53:22,199 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02104869975951674); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03262204056867377, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02104869975951674); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:22,200 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-09 12:53:22,200 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013677347617619715, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02104869975951674); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03262204056867377, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02104869975951674); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-09 12:53:22,433 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 364 ms to try 5000 different iterations.  Found a solution that moves 965 regions; Going from a computed imbalance of 0.013677347617619715 to a new imbalance of 6.017666210626442E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004774501003549676); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007075034750641397); MoveCostFunction : (multiplier=7.0, imbalance=0.10722222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004774501003549676); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:22,435 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1389499720:81 , srv1122854056:82 , srv1167585621:82 , srv53282203:82 , srv1409497334:83 , srv1544209436:83 , srv1644986708:83 , srv1746673477:83 , srv1916016982:83 , srv223154906:83 , srv111433994:84 , srv1129277650:84 , srv1424744835:84 , srv1555514162:84 , srv1652932689:84 , srv1688755249:84 , srv1917669639:84 , srv202620503:84 , srv2084589250:84 , srv219355945:84 , srv335579562:84 , srv757989544:84 , srv857599436:84 , srv891368953:84 , srv968475248:84 , srv1152381912:85 , srv1173987201:85 , srv1257926327:85 , srv1308220682:85 , srv1408472203:85 , srv1458739833:85 , srv1469652102:85 , srv1512898504:85 , srv1534555459:85 , srv1554645492:85 , srv1566315412:85 , srv1688095868:85 , srv1787907478:85 , srv1915580374:85 , srv1967016495:85 , srv245272677:85 , srv413801258:85 , srv424342086:85 , srv557106702:85 , srv870121093:85 , srv880309823:85 , srv904107282:85 , srv931322854:85 , srv1066358423:86 , srv1166626457:86 , srv1252876366:86 , srv1260944913:86 , srv1279546861:86 , srv1452502368:86 , srv1519458479:86 , srv1523688389:86 , srv1689173515:86 , srv1862676486:86 , srv1897455423:86 , srv2002506487:86 , srv2005036943:86 , srv2077960635:86 , srv2096836325:86 , srv2116079752:86 , srv239465894:86 , srv269956129:86 , srv304243223:86 , srv340745019:86 , srv389970669:86 , srv506724283:86 , srv785713128:86 , srv800176387:86 , srv860851164:86 , srv963845472:86 , srv1069413400:87 , srv1197215321:87 , srv1234966172:87 , srv1257027836:87 , srv1533058838:87 , srv1987881524:87 , srv1994833678:87 , srv2086385952:87 , srv245861660:87 , srv287798527:87 , srv291971214:87 , srv454853338:87 , srv511595041:87 , srv63837722:87 , srv640651210:87 , srv84232548:87 , srv855232642:87 , srv15895231:88 , srv1891001976:88 , srv2089267079:88 , srv212607282:88 , srv597350063:88 , srv948688173:88 , srv1335142388:89 , srv693624221:89 , srv227393303:539 }
2024-03-09 12:53:22,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:53:22,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167585621=7, srv860851164=91, srv1566315412=33, srv1452502368=22, srv2089267079=57, srv1279546861=15, srv1523688389=27, srv287798527=68, srv1746673477=40, srv2116079752=59, srv269956129=67, srv340745019=72, srv640651210=83, srv1257027836=12, srv1260944913=14, srv291971214=69, srv304243223=70, srv202620503=53, srv1688095868=37, srv1166626457=6, srv870121093=92, srv1409497334=20, srv1152381912=5, srv1994833678=50, srv1652932689=36, srv1891001976=43, srv880309823=93, srv968475248=99, srv2084589250=55, srv1122854056=3, srv1688755249=38, srv335579562=71, srv239465894=64, srv1916016982=46, srv963845472=98, srv1458739833=23, srv1197215321=9, srv800176387=87, srv223154906=62, srv63837722=82, srv111433994=2, srv693624221=84, srv2077960635=54, srv424342086=75, srv53282203=79, srv1554645492=31, srv1967016495=48, srv1408472203=19, srv245272677=65, srv557106702=80, srv1069413400=1, srv1917669639=47, srv84232548=88, srv1512898504=25, srv219355945=61, srv1555514162=32, srv2086385952=56, srv597350063=81, srv785713128=86, srv212607282=60, srv245861660=66, srv413801258=74, srv857599436=90, srv2002506487=51, srv1987881524=49, srv1234966172=10, srv1534555459=29, srv1862676486=42, srv2096836325=58, srv1424744835=21, srv511595041=78, srv855232642=89, srv891368953=94, srv757989544=85, srv904107282=95, srv1915580374=45, srv389970669=73, srv931322854=96, srv1066358423=0, srv1257926327=13, srv1173987201=8, srv1644986708=35, srv948688173=97, srv1519458479=26, srv1469652102=24, srv15895231=34, srv506724283=77, srv1897455423=44, srv1252876366=11, srv454853338=76, srv1533058838=28, srv1129277650=4, srv1689173515=39, srv1389499720=18, srv1787907478=41, srv1308220682=16, srv1335142388=17, srv1544209436=30, srv227393303=63, srv2005036943=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:53:22,450 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:53:22,451 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-09 12:53:22,452 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-09 12:53:22,453 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-09 12:53:22,454 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-09 12:53:22,455 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-09 12:53:22,456 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-09 12:53:22,557 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004774501003549676); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007075034750641397); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004774501003549676); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:22,557 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2024-03-09 12:53:22,557 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.950095466407967E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.004774501003549676); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007075034750641397); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.004774501003549676); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0035); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2024-03-09 12:53:22,880 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 416 ms to try 5000 different iterations.  Found a solution that moves 218 regions; Going from a computed imbalance of 5.950095466407967E-4 to a new imbalance of 1.886963505047328E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.02422222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:22,881 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1335142388:83 , srv1534555459:83 , srv1987881524:83 , srv1994833678:83 , srv245272677:83 , srv640651210:83 , srv693624221:83 , srv948688173:83 , srv1066358423:84 , srv1122854056:84 , srv1166626457:84 , srv1257027836:84 , srv1308220682:84 , srv1409497334:84 , srv1512898504:84 , srv1519458479:84 , srv1523688389:84 , srv1533058838:84 , srv1555514162:84 , srv1566315412:84 , srv1644986708:84 , srv1688095868:84 , srv1688755249:84 , srv1689173515:84 , srv1787907478:84 , srv1891001976:84 , srv1915580374:84 , srv1967016495:84 , srv2005036943:84 , srv202620503:84 , srv2086385952:84 , srv2089267079:84 , srv2096836325:84 , srv219355945:84 , srv239465894:84 , srv245861660:84 , srv269956129:84 , srv287798527:84 , srv304243223:84 , srv424342086:84 , srv454853338:84 , srv511595041:84 , srv53282203:84 , srv597350063:84 , srv63837722:84 , srv757989544:84 , srv785713128:84 , srv800176387:84 , srv857599436:84 , srv880309823:84 , srv891368953:84 , srv931322854:84 , srv963845472:84 , srv968475248:84 , srv1069413400:85 , srv1129277650:85 , srv1152381912:85 , srv1167585621:85 , srv1173987201:85 , srv1197215321:85 , srv1234966172:85 , srv1252876366:85 , srv1257926327:85 , srv1279546861:85 , srv1408472203:85 , srv1424744835:85 , srv1452502368:85 , srv1458739833:85 , srv1469652102:85 , srv1544209436:85 , srv1554645492:85 , srv15895231:85 , srv1652932689:85 , srv1862676486:85 , srv1897455423:85 , srv1916016982:85 , srv2002506487:85 , srv2077960635:85 , srv2116079752:85 , srv212607282:85 , srv223154906:85 , srv291971214:85 , srv335579562:85 , srv340745019:85 , srv389970669:85 , srv413801258:85 , srv506724283:85 , srv557106702:85 , srv84232548:85 , srv855232642:85 , srv860851164:85 , srv870121093:85 , srv904107282:85 , srv111433994:86 , srv1260944913:86 , srv1389499720:86 , srv1746673477:86 , srv1917669639:86 , srv2084589250:86 , srv227393303:641 }
2024-03-09 12:53:22,884 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:53:22,885 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1167585621=7, srv860851164=91, srv1566315412=33, srv1452502368=22, srv2089267079=57, srv1279546861=15, srv1523688389=27, srv287798527=68, srv1746673477=40, srv2116079752=59, srv269956129=67, srv340745019=72, srv640651210=83, srv1257027836=12, srv1260944913=14, srv291971214=69, srv304243223=70, srv202620503=53, srv1688095868=37, srv1166626457=6, srv870121093=92, srv1409497334=20, srv1152381912=5, srv1994833678=50, srv1652932689=36, srv1891001976=43, srv880309823=93, srv968475248=99, srv2084589250=55, srv1122854056=3, srv1688755249=38, srv335579562=71, srv239465894=64, srv1916016982=46, srv963845472=98, srv1458739833=23, srv1197215321=9, srv800176387=87, srv223154906=62, srv63837722=82, srv111433994=2, srv693624221=84, srv2077960635=54, srv424342086=75, srv53282203=79, srv1554645492=31, srv1967016495=48, srv1408472203=19, srv245272677=65, srv557106702=80, srv1069413400=1, srv1917669639=47, srv84232548=88, srv1512898504=25, srv219355945=61, srv1555514162=32, srv2086385952=56, srv597350063=81, srv785713128=86, srv212607282=60, srv245861660=66, srv413801258=74, srv857599436=90, srv2002506487=51, srv1987881524=49, srv1234966172=10, srv1534555459=29, srv1862676486=42, srv2096836325=58, srv1424744835=21, srv511595041=78, srv855232642=89, srv891368953=94, srv757989544=85, srv904107282=95, srv1915580374=45, srv389970669=73, srv931322854=96, srv1066358423=0, srv1257926327=13, srv1173987201=8, srv1644986708=35, srv948688173=97, srv1519458479=26, srv1469652102=24, srv15895231=34, srv506724283=77, srv1897455423=44, srv1252876366=11, srv454853338=76, srv1533058838=28, srv1129277650=4, srv1689173515=39, srv1389499720=18, srv1787907478=41, srv1308220682=16, srv1335142388=17, srv1544209436=30, srv227393303=63, srv2005036943=52} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2024-03-09 12:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2024-03-09 12:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2024-03-09 12:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2024-03-09 12:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2024-03-09 12:53:22,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2024-03-09 12:53:22,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2024-03-09 12:53:22,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2024-03-09 12:53:22,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2024-03-09 12:53:22,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2024-03-09 12:53:22,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2024-03-09 12:53:23,005 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.734316642460514E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:23,006 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1335142388:83 , srv1534555459:83 , srv1987881524:83 , srv1994833678:83 , srv245272677:83 , srv640651210:83 , srv693624221:83 , srv948688173:83 , srv1066358423:84 , srv1122854056:84 , srv1166626457:84 , srv1257027836:84 , srv1308220682:84 , srv1409497334:84 , srv1512898504:84 , srv1519458479:84 , srv1523688389:84 , srv1533058838:84 , srv1555514162:84 , srv1566315412:84 , srv1644986708:84 , srv1688095868:84 , srv1688755249:84 , srv1689173515:84 , srv1787907478:84 , srv1891001976:84 , srv1915580374:84 , srv1967016495:84 , srv2005036943:84 , srv202620503:84 , srv2086385952:84 , srv2089267079:84 , srv2096836325:84 , srv219355945:84 , srv239465894:84 , srv245861660:84 , srv269956129:84 , srv287798527:84 , srv304243223:84 , srv424342086:84 , srv454853338:84 , srv511595041:84 , srv53282203:84 , srv597350063:84 , srv63837722:84 , srv757989544:84 , srv785713128:84 , srv800176387:84 , srv857599436:84 , srv880309823:84 , srv891368953:84 , srv931322854:84 , srv963845472:84 , srv968475248:84 , srv1069413400:85 , srv1129277650:85 , srv1152381912:85 , srv1167585621:85 , srv1173987201:85 , srv1197215321:85 , srv1234966172:85 , srv1252876366:85 , srv1257926327:85 , srv1279546861:85 , srv1408472203:85 , srv1424744835:85 , srv1452502368:85 , srv1458739833:85 , srv1469652102:85 , srv1544209436:85 , srv1554645492:85 , srv15895231:85 , srv1652932689:85 , srv1862676486:85 , srv1897455423:85 , srv1916016982:85 , srv2002506487:85 , srv2077960635:85 , srv2116079752:85 , srv212607282:85 , srv223154906:85 , srv291971214:85 , srv335579562:85 , srv340745019:85 , srv389970669:85 , srv413801258:85 , srv506724283:85 , srv557106702:85 , srv84232548:85 , srv855232642:85 , srv860851164:85 , srv870121093:85 , srv904107282:85 , srv111433994:86 , srv1260944913:86 , srv1389499720:86 , srv1746673477:86 , srv1917669639:86 , srv2084589250:86 , srv227393303:641 }
2024-03-09 12:53:23,036 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 271), ProcessCount=171 (was 171), AvailableMemoryMB=11036 (was 11772)
2024-03-09 12:53:23,045 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=171, AvailableMemoryMB=11035
2024-03-09 12:53:23,046 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2024-03-09 12:53:23,046 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2024-03-09 12:53:23,055 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1032404919:84 , srv1102693512:84 , srv1107269963:84 , srv1182553228:84 , srv130642616:84 , srv1342442764:84 , srv1476070709:84 , srv1508658783:84 , srv1512428464:84 , srv1531335875:84 , srv157204580:84 , srv1745243394:84 , srv1745948701:84 , srv1882072337:84 , srv1941578071:84 , srv1945559476:84 , srv1961099247:84 , srv2017808404:84 , srv2047238643:84 , srv2133123485:84 , srv282980479:84 , srv300378712:84 , srv343367615:84 , srv440939923:84 , srv574591030:84 , srv601470108:84 , srv785102601:84 , srv79483980:84 , srv947455373:84 , srv2061768744:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2024-03-09 12:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:53:23,056 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2061768744=19, srv1102693512=1, srv1508658783=7, srv1745243394=11, srv2047238643=18, srv300378712=22, srv1032404919=0, srv157204580=10, srv947455373=29, srv343367615=23, srv2133123485=20, srv1961099247=16, srv282980479=21, srv1941578071=14, srv1476070709=6, srv1107269963=2, srv1182553228=3, srv574591030=25, srv130642616=4, srv1342442764=5, srv1531335875=9, srv1882072337=13, srv440939923=24, srv1945559476=15, srv79483980=28, srv1512428464=8, srv785102601=27, srv601470108=26, srv1745948701=12, srv2017808404=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:53:23,064 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:53:23,065 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:53:23,066 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-09 12:53:23,082 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1528470603860401, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:53:23,082 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.991267465936064, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1528470603860401, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2024-03-09 12:55:23,069 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6080672 different iterations.  Found a solution that moves 1859 regions; Going from a computed imbalance of 0.991267465936064 to a new imbalance of 4.338998739279625E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6885185185185185, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:55:23,076 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1032404919:90 , srv1102693512:90 , srv1107269963:90 , srv1182553228:90 , srv130642616:90 , srv1342442764:90 , srv1476070709:90 , srv1508658783:90 , srv1512428464:90 , srv1531335875:90 , srv157204580:90 , srv1745243394:90 , srv1745948701:90 , srv1882072337:90 , srv1941578071:90 , srv1945559476:90 , srv1961099247:90 , srv2017808404:90 , srv2047238643:90 , srv2061768744:90 , srv2133123485:90 , srv282980479:90 , srv300378712:90 , srv343367615:90 , srv440939923:90 , srv574591030:90 , srv601470108:90 , srv785102601:90 , srv79483980:90 , srv947455373:90 }
2024-03-09 12:55:23,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2024-03-09 12:55:23,077 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2061768744=19, srv1102693512=1, srv1508658783=7, srv1745243394=11, srv2047238643=18, srv300378712=22, srv1032404919=0, srv157204580=10, srv947455373=29, srv343367615=23, srv2133123485=20, srv1961099247=16, srv282980479=21, srv1941578071=14, srv1476070709=6, srv1107269963=2, srv1182553228=3, srv574591030=25, srv130642616=4, srv1342442764=5, srv1531335875=9, srv1882072337=13, srv440939923=24, srv1945559476=15, srv79483980=28, srv1512428464=8, srv785102601=27, srv601470108=26, srv1745948701=12, srv2017808404=17} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2024-03-09 12:55:23,081 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2024-03-09 12:55:23,082 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2024-03-09 12:55:23,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2024-03-09 12:55:23,094 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1032404919:90 , srv1102693512:90 , srv1107269963:90 , srv1182553228:90 , srv130642616:90 , srv1342442764:90 , srv1476070709:90 , srv1508658783:90 , srv1512428464:90 , srv1531335875:90 , srv157204580:90 , srv1745243394:90 , srv1745948701:90 , srv1882072337:90 , srv1941578071:90 , srv1945559476:90 , srv1961099247:90 , srv2017808404:90 , srv2047238643:90 , srv2061768744:90 , srv2133123485:90 , srv282980479:90 , srv300378712:90 , srv343367615:90 , srv440939923:90 , srv574591030:90 , srv601470108:90 , srv785102601:90 , srv79483980:90 , srv947455373:90 }
2024-03-09 12:55:23,111 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=179 (was 271), ProcessCount=170 (was 171), AvailableMemoryMB=10289 (was 11035)