2023-12-28 12:53:41,127 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-12-28 12:53:41,362 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-12-28 12:53:41,455 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-12-28 12:53:41,513 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-28 12:53:41,520 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-28 12:53:41,531 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=169, AvailableMemoryMB=15420
2023-12-28 12:53:41,533 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-28 12:53:41,534 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-12-28 12:53:41,651 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1038276666:84 , srv1056352673:84 , srv1057810930:84 , srv1070526057:84 , srv1086457043:84 , srv1211346170:84 , srv1253167125:84 , srv1265962980:84 , srv1266565017:84 , srv127346707:84 , srv129887356:84 , srv1304891167:84 , srv131030510:84 , srv1361102455:84 , srv1438935794:84 , srv1456242054:84 , srv1462296570:84 , srv1529526303:84 , srv1558564505:84 , srv1567128100:84 , srv1569326624:84 , srv1611453258:84 , srv1636326075:84 , srv1638092875:84 , srv1664756164:84 , srv1714313616:84 , srv1718548111:84 , srv1748222115:84 , srv1764316127:84 , srv1771334109:84 , srv1774192587:84 , srv1779890143:84 , srv1831287308:84 , srv1885169947:84 , srv1895782989:84 , srv1896806014:84 , srv1921977946:84 , srv1957374608:84 , srv1957546391:84 , srv2033096482:84 , srv2034337207:84 , srv205179425:84 , srv2069462919:84 , srv2093443503:84 , srv2110547148:84 , srv2111293914:84 , srv2124540255:84 , srv2128675109:84 , srv2137868151:84 , srv2146565218:84 , srv218994740:84 , srv220212462:84 , srv263282030:84 , srv267800382:84 , srv325025159:84 , srv332587468:84 , srv398067985:84 , srv407655928:84 , srv41225240:84 , srv416597058:84 , srv422325896:84 , srv44521740:84 , srv458568301:84 , srv478596246:84 , srv498379071:84 , srv500348410:84 , srv519084873:84 , srv574293588:84 , srv595945320:84 , srv596588099:84 , srv604498829:84 , srv610088755:84 , srv612677558:84 , srv668083522:84 , srv678579307:84 , srv696854309:84 , srv700213715:84 , srv719848119:84 , srv761956583:84 , srv770873756:84 , srv780685026:84 , srv802740115:84 , srv809578414:84 , srv814233935:84 , srv814426946:84 , srv818550384:84 , srv821467841:84 , srv856136750:84 , srv870716543:84 , srv877794879:84 , srv881515207:84 , srv888161838:84 , srv893487521:84 , srv921405910:84 , srv928777282:84 , srv961622529:84 , srv964975046:84 , srv983546499:84 , srv996314580:84 , srv1541370526:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-12-28 12:53:41,658 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:53:41,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1764316127=29, srv422325896=61, srv1774192587=31, srv1957374608=38, srv220212462=52, srv129887356=10, srv780685026=81, srv1541370526=18, srv131030510=12, srv127346707=9, srv1779890143=32, srv500348410=66, srv1638092875=24, srv1456242054=15, srv983546499=98, srv1957546391=39, srv921405910=94, srv1361102455=13, srv332587468=56, srv2110547148=45, srv877794879=90, srv1664756164=25, srv407655928=58, srv719848119=78, srv2093443503=44, srv1086457043=4, srv2069462919=43, srv458568301=63, srv41225240=59, srv870716543=89, srv1558564505=19, srv1718548111=27, srv1253167125=6, srv596588099=70, srv267800382=54, srv888161838=92, srv2128675109=48, srv996314580=99, srv1266565017=8, srv205179425=42, srv1636326075=23, srv881515207=91, srv1895782989=35, srv1304891167=11, srv961622529=96, srv668083522=74, srv856136750=88, srv1070526057=3, srv44521740=62, srv1611453258=22, srv1896806014=36, srv1921977946=37, srv1569326624=21, srv595945320=69, srv814233935=84, srv612677558=73, srv2034337207=41, srv818550384=86, srv604498829=71, srv2124540255=47, srv802740115=82, srv821467841=87, srv498379071=65, srv1057810930=2, srv1748222115=28, srv610088755=72, srv809578414=83, srv1038276666=0, srv1831287308=33, srv218994740=51, srv519084873=67, srv1714313616=26, srv574293588=68, srv964975046=97, srv1567128100=20, srv678579307=75, srv761956583=79, srv2146565218=50, srv1265962980=7, srv893487521=93, srv263282030=53, srv1056352673=1, srv770873756=80, srv928777282=95, srv700213715=77, srv1771334109=30, srv1211346170=5, srv1529526303=17, srv416597058=60, srv1885169947=34, srv2137868151=49, srv1462296570=16, srv325025159=55, srv1438935794=14, srv2111293914=46, srv814426946=85, srv478596246=64, srv696854309=76, srv2033096482=40, srv398067985=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:53:41,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:53:41,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:53:41,726 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:53:41,727 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-28 12:53:41,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-28 12:53:41,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-28 12:53:41,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-28 12:53:41,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-28 12:53:41,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-28 12:53:41,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-28 12:53:41,729 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-28 12:53:41,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-28 12:53:41,731 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-28 12:53:41,732 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-28 12:53:41,733 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-28 12:53:41,734 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-28 12:53:41,735 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-28 12:53:41,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-28 12:53:41,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-28 12:53:41,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-28 12:53:41,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-28 12:53:41,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-28 12:53:41,736 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-28 12:53:41,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-28 12:53:41,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-28 12:53:41,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-28 12:53:41,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-28 12:53:41,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-28 12:53:41,885 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:41,886 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-28 12:53:41,886 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-28 12:53:42,362 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 604 ms to try 5000 different iterations.  Found a solution that moves 4647 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.185403901784537. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04294757648547596, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05836915686923309, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5163333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04294757648547596, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18066666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24716666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:42,372 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1636326075:82 , srv1771334109:82 , srv2034337207:82 , srv1529526303:83 , srv1638092875:83 , srv1774192587:83 , srv416597058:83 , srv458568301:83 , srv821467841:83 , srv921405910:83 , srv1038276666:84 , srv1714313616:84 , srv1779890143:84 , srv332587468:84 , srv41225240:84 , srv422325896:84 , srv478596246:84 , srv596588099:84 , srv700213715:84 , srv719848119:84 , srv770873756:84 , srv809578414:84 , srv818550384:84 , srv928777282:84 , srv1265962980:85 , srv1304891167:85 , srv1438935794:85 , srv1456242054:85 , srv1462296570:85 , srv1569326624:85 , srv1764316127:85 , srv1885169947:85 , srv1896806014:85 , srv1921977946:85 , srv1957374608:85 , srv205179425:85 , srv2093443503:85 , srv407655928:85 , srv612677558:85 , srv678579307:85 , srv696854309:85 , srv761956583:85 , srv780685026:85 , srv856136750:85 , srv881515207:85 , srv131030510:86 , srv1567128100:86 , srv1611453258:86 , srv2146565218:86 , srv218994740:86 , srv267800382:86 , srv574293588:86 , srv604498829:86 , srv610088755:86 , srv814233935:86 , srv870716543:86 , srv964975046:86 , srv983546499:86 , srv996314580:86 , srv1086457043:87 , srv127346707:87 , srv1558564505:87 , srv1718548111:87 , srv1895782989:87 , srv2033096482:87 , srv2069462919:87 , srv2111293914:87 , srv2124540255:87 , srv220212462:87 , srv263282030:87 , srv595945320:87 , srv961622529:87 , srv1057810930:88 , srv1070526057:88 , srv1211346170:88 , srv1748222115:88 , srv1831287308:88 , srv1957546391:88 , srv2110547148:88 , srv2137868151:88 , srv325025159:88 , srv44521740:88 , srv498379071:88 , srv519084873:88 , srv888161838:88 , srv1056352673:89 , srv1253167125:89 , srv1266565017:89 , srv129887356:89 , srv1664756164:89 , srv802740115:89 , srv814426946:89 , srv1361102455:90 , srv668083522:90 , srv877794879:90 , srv398067985:91 , srv893487521:91 , srv2128675109:92 , srv500348410:94 , srv1541370526:472 }
2023-12-28 12:53:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:53:42,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1764316127=29, srv422325896=61, srv1774192587=31, srv1957374608=38, srv220212462=52, srv129887356=10, srv780685026=81, srv1541370526=18, srv131030510=12, srv127346707=9, srv1779890143=32, srv500348410=66, srv1638092875=24, srv1456242054=15, srv983546499=98, srv1957546391=39, srv921405910=94, srv1361102455=13, srv332587468=56, srv2110547148=45, srv877794879=90, srv1664756164=25, srv407655928=58, srv719848119=78, srv2093443503=44, srv1086457043=4, srv2069462919=43, srv458568301=63, srv41225240=59, srv870716543=89, srv1558564505=19, srv1718548111=27, srv1253167125=6, srv596588099=70, srv267800382=54, srv888161838=92, srv2128675109=48, srv996314580=99, srv1266565017=8, srv205179425=42, srv1636326075=23, srv881515207=91, srv1895782989=35, srv1304891167=11, srv961622529=96, srv668083522=74, srv856136750=88, srv1070526057=3, srv44521740=62, srv1611453258=22, srv1896806014=36, srv1921977946=37, srv1569326624=21, srv595945320=69, srv814233935=84, srv612677558=73, srv2034337207=41, srv818550384=86, srv604498829=71, srv2124540255=47, srv802740115=82, srv821467841=87, srv498379071=65, srv1057810930=2, srv1748222115=28, srv610088755=72, srv809578414=83, srv1038276666=0, srv1831287308=33, srv218994740=51, srv519084873=67, srv1714313616=26, srv574293588=68, srv964975046=97, srv1567128100=20, srv678579307=75, srv761956583=79, srv2146565218=50, srv1265962980=7, srv893487521=93, srv263282030=53, srv1056352673=1, srv770873756=80, srv928777282=95, srv700213715=77, srv1771334109=30, srv1211346170=5, srv1529526303=17, srv416597058=60, srv1885169947=34, srv2137868151=49, srv1462296570=16, srv325025159=55, srv1438935794=14, srv2111293914=46, srv814426946=85, srv478596246=64, srv696854309=76, srv2033096482=40, srv398067985=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:53:42,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:53:42,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:53:42,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:53:42,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:53:42,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:53:42,396 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:53:42,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-28 12:53:42,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-28 12:53:42,399 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-28 12:53:42,400 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-28 12:53:42,401 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-28 12:53:42,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-28 12:53:42,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-28 12:53:42,508 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04294757648547596, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05836915686923309, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04294757648547596, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18066666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24716666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:42,508 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-28 12:53:42,508 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18537136279506722, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04294757648547596, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05836915686923309, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04294757648547596, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.18066666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24716666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-28 12:53:42,689 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 270 ms to try 5000 different iterations.  Found a solution that moves 2648 regions; Going from a computed imbalance of 0.18537136279506722 to a new imbalance of 0.016602019497442777. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022153654034508242); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035381518506868125, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2942222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022153654034508242); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01375); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:42,697 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv500348410:78 , srv893487521:79 , srv1664756164:81 , srv877794879:81 , srv1831287308:82 , srv2128675109:82 , srv668083522:82 , srv802740115:82 , srv814426946:82 , srv1211346170:83 , srv1895782989:83 , srv1957546391:83 , srv2124540255:83 , srv398067985:83 , srv44521740:83 , srv498379071:83 , srv1056352673:84 , srv1070526057:84 , srv1086457043:84 , srv1253167125:84 , srv1266565017:84 , srv1361102455:84 , srv2110547148:84 , srv220212462:84 , srv263282030:84 , srv325025159:84 , srv595945320:84 , srv1265962980:85 , srv129887356:85 , srv1748222115:85 , srv2033096482:85 , srv2111293914:85 , srv2137868151:85 , srv2146565218:85 , srv267800382:85 , srv407655928:85 , srv519084873:85 , srv814233935:85 , srv856136750:85 , srv888161838:85 , srv961622529:85 , srv1057810930:86 , srv1304891167:86 , srv1438935794:86 , srv1462296570:86 , srv1558564505:86 , srv1718548111:86 , srv1896806014:86 , srv1921977946:86 , srv205179425:86 , srv2069462919:86 , srv2093443503:86 , srv41225240:86 , srv610088755:86 , srv678579307:86 , srv780685026:86 , srv881515207:86 , srv964975046:86 , srv983546499:86 , srv996314580:86 , srv131030510:87 , srv1567128100:87 , srv1569326624:87 , srv1611453258:87 , srv1714313616:87 , srv1779890143:87 , srv1957374608:87 , srv574293588:87 , srv696854309:87 , srv719848119:87 , srv761956583:87 , srv770873756:87 , srv870716543:87 , srv928777282:87 , srv127346707:88 , srv1456242054:88 , srv1529526303:88 , srv1764316127:88 , srv1771334109:88 , srv1885169947:88 , srv218994740:88 , srv478596246:88 , srv612677558:88 , srv700213715:88 , srv1638092875:89 , srv1774192587:89 , srv422325896:89 , srv596588099:89 , srv604498829:89 , srv809578414:89 , srv818550384:89 , srv921405910:89 , srv2034337207:90 , srv332587468:90 , srv416597058:90 , srv458568301:90 , srv1038276666:91 , srv1636326075:91 , srv821467841:91 , srv1541370526:499 }
2023-12-28 12:53:42,700 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:53:42,701 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1764316127=29, srv422325896=61, srv1774192587=31, srv1957374608=38, srv220212462=52, srv129887356=10, srv780685026=81, srv1541370526=18, srv131030510=12, srv127346707=9, srv1779890143=32, srv500348410=66, srv1638092875=24, srv1456242054=15, srv983546499=98, srv1957546391=39, srv921405910=94, srv1361102455=13, srv332587468=56, srv2110547148=45, srv877794879=90, srv1664756164=25, srv407655928=58, srv719848119=78, srv2093443503=44, srv1086457043=4, srv2069462919=43, srv458568301=63, srv41225240=59, srv870716543=89, srv1558564505=19, srv1718548111=27, srv1253167125=6, srv596588099=70, srv267800382=54, srv888161838=92, srv2128675109=48, srv996314580=99, srv1266565017=8, srv205179425=42, srv1636326075=23, srv881515207=91, srv1895782989=35, srv1304891167=11, srv961622529=96, srv668083522=74, srv856136750=88, srv1070526057=3, srv44521740=62, srv1611453258=22, srv1896806014=36, srv1921977946=37, srv1569326624=21, srv595945320=69, srv814233935=84, srv612677558=73, srv2034337207=41, srv818550384=86, srv604498829=71, srv2124540255=47, srv802740115=82, srv821467841=87, srv498379071=65, srv1057810930=2, srv1748222115=28, srv610088755=72, srv809578414=83, srv1038276666=0, srv1831287308=33, srv218994740=51, srv519084873=67, srv1714313616=26, srv574293588=68, srv964975046=97, srv1567128100=20, srv678579307=75, srv761956583=79, srv2146565218=50, srv1265962980=7, srv893487521=93, srv263282030=53, srv1056352673=1, srv770873756=80, srv928777282=95, srv700213715=77, srv1771334109=30, srv1211346170=5, srv1529526303=17, srv416597058=60, srv1885169947=34, srv2137868151=49, srv1462296570=16, srv325025159=55, srv1438935794=14, srv2111293914=46, srv814426946=85, srv478596246=64, srv696854309=76, srv2033096482=40, srv398067985=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:53:42,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:53:42,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:53:42,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:53:42,718 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:53:42,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-28 12:53:42,720 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-28 12:53:42,721 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-28 12:53:42,722 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-28 12:53:42,723 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-28 12:53:42,724 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-28 12:53:42,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-28 12:53:42,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-28 12:53:42,725 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-28 12:53:42,814 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022153654034508242); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035381518506868125, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022153654034508242); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01375); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:42,814 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-28 12:53:42,815 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016583477805143243, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022153654034508242); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.035381518506868125, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022153654034508242); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01375); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04375, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-28 12:53:42,951 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 215 ms to try 5000 different iterations.  Found a solution that moves 990 regions; Going from a computed imbalance of 0.016583477805143243 to a new imbalance of 7.914817382911948E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005001558554945742); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005605833101216811); MoveCostFunction : (multiplier=7.0, imbalance=0.11, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005001558554945742); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:42,953 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1038276666:82 , srv127346707:82 , srv604498829:82 , srv129887356:83 , srv1456242054:83 , srv218994740:83 , srv332587468:83 , srv398067985:83 , srv1057810930:84 , srv131030510:84 , srv1567128100:84 , srv1611453258:84 , srv1748222115:84 , srv1764316127:84 , srv1885169947:84 , srv2128675109:84 , srv416597058:84 , srv458568301:84 , srv574293588:84 , srv821467841:84 , srv1056352673:85 , srv1070526057:85 , srv1266565017:85 , srv1361102455:85 , srv1558564505:85 , srv1636326075:85 , srv1638092875:85 , srv1774192587:85 , srv2111293914:85 , srv2137868151:85 , srv325025159:85 , srv422325896:85 , srv519084873:85 , srv596588099:85 , srv610088755:85 , srv696854309:85 , srv761956583:85 , srv818550384:85 , srv870716543:85 , srv888161838:85 , srv961622529:85 , srv983546499:85 , srv1086457043:86 , srv1253167125:86 , srv1438935794:86 , srv1462296570:86 , srv1529526303:86 , srv1569326624:86 , srv1714313616:86 , srv1718548111:86 , srv1779890143:86 , srv1896806014:86 , srv1957374608:86 , srv2033096482:86 , srv2034337207:86 , srv205179425:86 , srv2069462919:86 , srv2093443503:86 , srv2146565218:86 , srv220212462:86 , srv263282030:86 , srv267800382:86 , srv44521740:86 , srv500348410:86 , srv595945320:86 , srv612677558:86 , srv668083522:86 , srv678579307:86 , srv719848119:86 , srv780685026:86 , srv802740115:86 , srv809578414:86 , srv814426946:86 , srv928777282:86 , srv996314580:86 , srv1211346170:87 , srv1265962980:87 , srv1304891167:87 , srv1895782989:87 , srv1921977946:87 , srv2110547148:87 , srv2124540255:87 , srv407655928:87 , srv41225240:87 , srv478596246:87 , srv498379071:87 , srv700213715:87 , srv770873756:87 , srv814233935:87 , srv856136750:87 , srv877794879:87 , srv881515207:87 , srv893487521:87 , srv921405910:87 , srv964975046:87 , srv1664756164:88 , srv1771334109:88 , srv1831287308:88 , srv1957546391:88 , srv1541370526:531 }
2023-12-28 12:53:42,956 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:53:42,957 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1764316127=29, srv422325896=61, srv1774192587=31, srv1957374608=38, srv220212462=52, srv129887356=10, srv780685026=81, srv1541370526=18, srv131030510=12, srv127346707=9, srv1779890143=32, srv500348410=66, srv1638092875=24, srv1456242054=15, srv983546499=98, srv1957546391=39, srv921405910=94, srv1361102455=13, srv332587468=56, srv2110547148=45, srv877794879=90, srv1664756164=25, srv407655928=58, srv719848119=78, srv2093443503=44, srv1086457043=4, srv2069462919=43, srv458568301=63, srv41225240=59, srv870716543=89, srv1558564505=19, srv1718548111=27, srv1253167125=6, srv596588099=70, srv267800382=54, srv888161838=92, srv2128675109=48, srv996314580=99, srv1266565017=8, srv205179425=42, srv1636326075=23, srv881515207=91, srv1895782989=35, srv1304891167=11, srv961622529=96, srv668083522=74, srv856136750=88, srv1070526057=3, srv44521740=62, srv1611453258=22, srv1896806014=36, srv1921977946=37, srv1569326624=21, srv595945320=69, srv814233935=84, srv612677558=73, srv2034337207=41, srv818550384=86, srv604498829=71, srv2124540255=47, srv802740115=82, srv821467841=87, srv498379071=65, srv1057810930=2, srv1748222115=28, srv610088755=72, srv809578414=83, srv1038276666=0, srv1831287308=33, srv218994740=51, srv519084873=67, srv1714313616=26, srv574293588=68, srv964975046=97, srv1567128100=20, srv678579307=75, srv761956583=79, srv2146565218=50, srv1265962980=7, srv893487521=93, srv263282030=53, srv1056352673=1, srv770873756=80, srv928777282=95, srv700213715=77, srv1771334109=30, srv1211346170=5, srv1529526303=17, srv416597058=60, srv1885169947=34, srv2137868151=49, srv1462296570=16, srv325025159=55, srv1438935794=14, srv2111293914=46, srv814426946=85, srv478596246=64, srv696854309=76, srv2033096482=40, srv398067985=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:53:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:53:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-28 12:53:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-28 12:53:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-28 12:53:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-28 12:53:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-28 12:53:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-28 12:53:43,056 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005001558554945742); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005605833101216811); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005001558554945742); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:43,057 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-12-28 12:53:43,057 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.845496101278486E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005001558554945742); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005605833101216811); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005001558554945742); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=4.166666666666667E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-12-28 12:53:43,257 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 277 ms to try 5000 different iterations.  Found a solution that moves 222 regions; Going from a computed imbalance of 7.845496101278486E-4 to a new imbalance of 1.8758691399634464E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.024666666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:43,258 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1253167125:83 , srv1361102455:83 , srv1664756164:83 , srv1718548111:83 , srv1957546391:83 , srv2069462919:83 , srv2110547148:83 , srv612677558:83 , srv700213715:83 , srv809578414:83 , srv921405910:83 , srv964975046:83 , srv1056352673:84 , srv1057810930:84 , srv1211346170:84 , srv1266565017:84 , srv1304891167:84 , srv1462296570:84 , srv1558564505:84 , srv1569326624:84 , srv1611453258:84 , srv1714313616:84 , srv1771334109:84 , srv1831287308:84 , srv1921977946:84 , srv1957374608:84 , srv2033096482:84 , srv2034337207:84 , srv2128675109:84 , srv2137868151:84 , srv422325896:84 , srv478596246:84 , srv498379071:84 , srv500348410:84 , srv574293588:84 , srv596588099:84 , srv668083522:84 , srv770873756:84 , srv802740115:84 , srv814233935:84 , srv818550384:84 , srv821467841:84 , srv870716543:84 , srv877794879:84 , srv881515207:84 , srv888161838:84 , srv983546499:84 , srv996314580:84 , srv1038276666:85 , srv1070526057:85 , srv1086457043:85 , srv1265962980:85 , srv127346707:85 , srv129887356:85 , srv131030510:85 , srv1438935794:85 , srv1529526303:85 , srv1567128100:85 , srv1636326075:85 , srv1748222115:85 , srv1764316127:85 , srv1774192587:85 , srv1779890143:85 , srv1885169947:85 , srv1895782989:85 , srv1896806014:85 , srv205179425:85 , srv2093443503:85 , srv2111293914:85 , srv2124540255:85 , srv2146565218:85 , srv218994740:85 , srv220212462:85 , srv263282030:85 , srv267800382:85 , srv332587468:85 , srv398067985:85 , srv407655928:85 , srv41225240:85 , srv416597058:85 , srv44521740:85 , srv458568301:85 , srv519084873:85 , srv595945320:85 , srv604498829:85 , srv610088755:85 , srv678579307:85 , srv696854309:85 , srv719848119:85 , srv761956583:85 , srv780685026:85 , srv814426946:85 , srv856136750:85 , srv893487521:85 , srv928777282:85 , srv1638092875:86 , srv325025159:86 , srv961622529:86 , srv1456242054:87 , srv1541370526:640 }
2023-12-28 12:53:43,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:53:43,262 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1764316127=29, srv422325896=61, srv1774192587=31, srv1957374608=38, srv220212462=52, srv129887356=10, srv780685026=81, srv1541370526=18, srv131030510=12, srv127346707=9, srv1779890143=32, srv500348410=66, srv1638092875=24, srv1456242054=15, srv983546499=98, srv1957546391=39, srv921405910=94, srv1361102455=13, srv332587468=56, srv2110547148=45, srv877794879=90, srv1664756164=25, srv407655928=58, srv719848119=78, srv2093443503=44, srv1086457043=4, srv2069462919=43, srv458568301=63, srv41225240=59, srv870716543=89, srv1558564505=19, srv1718548111=27, srv1253167125=6, srv596588099=70, srv267800382=54, srv888161838=92, srv2128675109=48, srv996314580=99, srv1266565017=8, srv205179425=42, srv1636326075=23, srv881515207=91, srv1895782989=35, srv1304891167=11, srv961622529=96, srv668083522=74, srv856136750=88, srv1070526057=3, srv44521740=62, srv1611453258=22, srv1896806014=36, srv1921977946=37, srv1569326624=21, srv595945320=69, srv814233935=84, srv612677558=73, srv2034337207=41, srv818550384=86, srv604498829=71, srv2124540255=47, srv802740115=82, srv821467841=87, srv498379071=65, srv1057810930=2, srv1748222115=28, srv610088755=72, srv809578414=83, srv1038276666=0, srv1831287308=33, srv218994740=51, srv519084873=67, srv1714313616=26, srv574293588=68, srv964975046=97, srv1567128100=20, srv678579307=75, srv761956583=79, srv2146565218=50, srv1265962980=7, srv893487521=93, srv263282030=53, srv1056352673=1, srv770873756=80, srv928777282=95, srv700213715=77, srv1771334109=30, srv1211346170=5, srv1529526303=17, srv416597058=60, srv1885169947=34, srv2137868151=49, srv1462296570=16, srv325025159=55, srv1438935794=14, srv2111293914=46, srv814426946=85, srv478596246=64, srv696854309=76, srv2033096482=40, srv398067985=57} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:53:43,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:53:43,272 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-12-28 12:53:43,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-12-28 12:53:43,274 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-12-28 12:53:43,275 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-12-28 12:53:43,276 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-12-28 12:53:43,277 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-12-28 12:53:43,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.7204214175126534E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0017082358106351274); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:43,358 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1253167125:83 , srv1361102455:83 , srv1664756164:83 , srv1718548111:83 , srv1957546391:83 , srv2069462919:83 , srv2110547148:83 , srv612677558:83 , srv700213715:83 , srv809578414:83 , srv921405910:83 , srv964975046:83 , srv1056352673:84 , srv1057810930:84 , srv1211346170:84 , srv1266565017:84 , srv1304891167:84 , srv1462296570:84 , srv1558564505:84 , srv1569326624:84 , srv1611453258:84 , srv1714313616:84 , srv1771334109:84 , srv1831287308:84 , srv1921977946:84 , srv1957374608:84 , srv2033096482:84 , srv2034337207:84 , srv2128675109:84 , srv2137868151:84 , srv422325896:84 , srv478596246:84 , srv498379071:84 , srv500348410:84 , srv574293588:84 , srv596588099:84 , srv668083522:84 , srv770873756:84 , srv802740115:84 , srv814233935:84 , srv818550384:84 , srv821467841:84 , srv870716543:84 , srv877794879:84 , srv881515207:84 , srv888161838:84 , srv983546499:84 , srv996314580:84 , srv1038276666:85 , srv1070526057:85 , srv1086457043:85 , srv1265962980:85 , srv127346707:85 , srv129887356:85 , srv131030510:85 , srv1438935794:85 , srv1529526303:85 , srv1567128100:85 , srv1636326075:85 , srv1748222115:85 , srv1764316127:85 , srv1774192587:85 , srv1779890143:85 , srv1885169947:85 , srv1895782989:85 , srv1896806014:85 , srv205179425:85 , srv2093443503:85 , srv2111293914:85 , srv2124540255:85 , srv2146565218:85 , srv218994740:85 , srv220212462:85 , srv263282030:85 , srv267800382:85 , srv332587468:85 , srv398067985:85 , srv407655928:85 , srv41225240:85 , srv416597058:85 , srv44521740:85 , srv458568301:85 , srv519084873:85 , srv595945320:85 , srv604498829:85 , srv610088755:85 , srv678579307:85 , srv696854309:85 , srv719848119:85 , srv761956583:85 , srv780685026:85 , srv814426946:85 , srv856136750:85 , srv893487521:85 , srv928777282:85 , srv1638092875:86 , srv325025159:86 , srv961622529:86 , srv1456242054:87 , srv1541370526:640 }
2023-12-28 12:53:43,379 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=337 (was 337), ProcessCount=169 (was 169), AvailableMemoryMB=14955 (was 15420)
2023-12-28 12:53:43,386 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=337, ProcessCount=169, AvailableMemoryMB=14955
2023-12-28 12:53:43,386 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-12-28 12:53:43,386 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-12-28 12:53:43,393 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1123760157:84 , srv1127015919:84 , srv1133445198:84 , srv1180191670:84 , srv12357876:84 , srv1244548277:84 , srv1272204414:84 , srv1353279128:84 , srv1452726351:84 , srv1620055844:84 , srv1912700913:84 , srv1932609454:84 , srv1982929924:84 , srv2085403699:84 , srv2128218125:84 , srv231165485:84 , srv289333013:84 , srv305415266:84 , srv492756488:84 , srv541688183:84 , srv562038162:84 , srv619752301:84 , srv628888840:84 , srv670105883:84 , srv759752600:84 , srv822344926:84 , srv832733868:84 , srv866952255:84 , srv891962785:84 , srv252298858:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-12-28 12:53:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:53:43,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2085403699=13, srv231165485=15, srv1127015919=1, srv1133445198=2, srv1452726351=8, srv492756488=19, srv619752301=22, srv832733868=27, srv1912700913=10, srv866952255=28, srv1982929924=12, srv1123760157=0, srv1932609454=11, srv759752600=25, srv1353279128=7, srv891962785=29, srv541688183=20, srv252298858=16, srv562038162=21, srv670105883=24, srv1272204414=6, srv12357876=4, srv628888840=23, srv1244548277=5, srv822344926=26, srv1620055844=9, srv305415266=18, srv289333013=17, srv1180191670=3, srv2128218125=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:53:43,402 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:53:43,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-28 12:53:43,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.079485608003372, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:53:43,417 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912443499819655, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.079485608003372, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-12-28 12:55:43,406 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 10549755 different iterations.  Found a solution that moves 1834 regions; Going from a computed imbalance of 0.9912443499819655 to a new imbalance of 4.280647492113412E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6792592592592592, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:55:43,413 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1123760157:90 , srv1127015919:90 , srv1133445198:90 , srv1180191670:90 , srv12357876:90 , srv1244548277:90 , srv1272204414:90 , srv1353279128:90 , srv1452726351:90 , srv1620055844:90 , srv1912700913:90 , srv1932609454:90 , srv1982929924:90 , srv2085403699:90 , srv2128218125:90 , srv231165485:90 , srv252298858:90 , srv289333013:90 , srv305415266:90 , srv492756488:90 , srv541688183:90 , srv562038162:90 , srv619752301:90 , srv628888840:90 , srv670105883:90 , srv759752600:90 , srv822344926:90 , srv832733868:90 , srv866952255:90 , srv891962785:90 }
2023-12-28 12:55:43,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-12-28 12:55:43,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv2085403699=13, srv231165485=15, srv1127015919=1, srv1133445198=2, srv1452726351=8, srv492756488=19, srv619752301=22, srv832733868=27, srv1912700913=10, srv866952255=28, srv1982929924=12, srv1123760157=0, srv1932609454=11, srv759752600=25, srv1353279128=7, srv891962785=29, srv541688183=20, srv252298858=16, srv562038162=21, srv670105883=24, srv1272204414=6, srv12357876=4, srv628888840=23, srv1244548277=5, srv822344926=26, srv1620055844=9, srv305415266=18, srv289333013=17, srv1180191670=3, srv2128218125=14} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-12-28 12:55:43,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-12-28 12:55:43,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-12-28 12:55:43,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-12-28 12:55:43,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-12-28 12:55:43,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-12-28 12:55:43,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-12-28 12:55:43,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-12-28 12:55:43,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-12-28 12:55:43,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-12-28 12:55:43,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-12-28 12:55:43,436 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-12-28 12:55:43,436 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1123760157:90 , srv1127015919:90 , srv1133445198:90 , srv1180191670:90 , srv12357876:90 , srv1244548277:90 , srv1272204414:90 , srv1353279128:90 , srv1452726351:90 , srv1620055844:90 , srv1912700913:90 , srv1932609454:90 , srv1982929924:90 , srv2085403699:90 , srv2128218125:90 , srv231165485:90 , srv252298858:90 , srv289333013:90 , srv305415266:90 , srv492756488:90 , srv541688183:90 , srv562038162:90 , srv619752301:90 , srv628888840:90 , srv670105883:90 , srv759752600:90 , srv822344926:90 , srv832733868:90 , srv866952255:90 , srv891962785:90 }
2023-12-28 12:55:43,449 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=174 (was 337), ProcessCount=169 (was 169), AvailableMemoryMB=14622 (was 14955)