2023-10-03 00:53:26,122 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-03 00:53:26,406 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-03 00:53:26,509 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-03 00:53:26,566 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-03 00:53:26,573 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-03 00:53:26,586 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=277, ProcessCount=171, AvailableMemoryMB=9533
2023-10-03 00:53:26,588 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-03 00:53:26,589 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-03 00:53:26,731 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100469462:84 , srv1009636015:84 , srv1009871517:84 , srv1029023651:84 , srv1030531371:84 , srv103373762:84 , srv1038917213:84 , srv1064254193:84 , srv1086231570:84 , srv1095153082:84 , srv1108457568:84 , srv1146345105:84 , srv1171403200:84 , srv1194811758:84 , srv1212679138:84 , srv1213775560:84 , srv1219016369:84 , srv1237337226:84 , srv1255693594:84 , srv1273737503:84 , srv1287465525:84 , srv1364757860:84 , srv1395643280:84 , srv1408742299:84 , srv143450436:84 , srv1452266744:84 , srv1458932542:84 , srv1462049003:84 , srv1467545817:84 , srv1512119709:84 , srv1525979068:84 , srv1527369217:84 , srv1542231432:84 , srv1566762735:84 , srv1592536003:84 , srv1629301089:84 , srv1632865003:84 , srv1634879736:84 , srv1661116087:84 , srv16779267:84 , srv1683164339:84 , srv1701527495:84 , srv1707946553:84 , srv171257298:84 , srv1744734291:84 , srv1747543675:84 , srv1769266985:84 , srv1826032875:84 , srv1829160154:84 , srv183758406:84 , srv1840713835:84 , srv1876200360:84 , srv188778820:84 , srv191162910:84 , srv1922897018:84 , srv1951705434:84 , srv199306962:84 , srv2014428848:84 , srv2015009032:84 , srv2019672894:84 , srv2029221234:84 , srv2037840305:84 , srv2055656395:84 , srv2073235983:84 , srv2090428:84 , srv2102541958:84 , srv211401934:84 , srv251234794:84 , srv268281654:84 , srv291159477:84 , srv31559955:84 , srv32424961:84 , srv327447716:84 , srv349918269:84 , srv416781929:84 , srv416815301:84 , srv420751634:84 , srv42171959:84 , srv511453099:84 , srv558798104:84 , srv569801155:84 , srv573429280:84 , srv575909051:84 , srv602926451:84 , srv604146866:84 , srv636415128:84 , srv666569593:84 , srv714331115:84 , srv724187010:84 , srv743148795:84 , srv775106451:84 , srv790092678:84 , srv798869826:84 , srv806253023:84 , srv918645950:84 , srv93610488:84 , srv974687775:84 , srv981218582:84 , srv984589955:84 , srv1472164647:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-03 00:53:26,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:26,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv790092678=92, srv1632865003=37, srv349918269=74, srv1009871517=2, srv191162910=54, srv2029221234=61, srv1826032875=48, srv188778820=53, srv1146345105=11, srv171257298=44, srv268281654=69, srv199306962=57, srv1095153082=9, srv183758406=50, srv291159477=70, srv1287465525=20, srv602926451=84, srv1452266744=25, srv1395643280=22, srv1467545817=28, srv143450436=24, srv1525979068=31, srv573429280=82, srv1829160154=49, srv93610488=96, srv1194811758=13, srv211401934=67, srv743148795=90, srv1629301089=36, srv1472164647=29, srv416781929=75, srv918645950=95, srv1634879736=38, srv1255693594=18, srv1683164339=41, srv1212679138=14, srv1707946553=43, srv1542231432=33, srv2014428848=58, srv714331115=88, srv806253023=94, srv1108457568=10, srv1876200360=52, srv1951705434=56, srv327447716=73, srv984589955=99, srv31559955=71, srv1237337226=17, srv1769266985=47, srv775106451=91, srv1064254193=7, srv416815301=76, srv16779267=40, srv1566762735=34, srv100469462=0, srv2037840305=62, srv1592536003=35, srv2102541958=66, srv974687775=97, srv42171959=78, srv1922897018=55, srv2019672894=60, srv251234794=68, srv1512119709=30, srv420751634=77, srv558798104=80, srv1219016369=16, srv1408742299=23, srv1458932542=26, srv1744734291=45, srv1030531371=4, srv666569593=87, srv103373762=5, srv2015009032=59, srv2073235983=64, srv604146866=85, srv1171403200=12, srv1273737503=19, srv575909051=83, srv636415128=86, srv1701527495=42, srv1840713835=51, srv1029023651=3, srv1462049003=27, srv1086231570=8, srv1038917213=6, srv1364757860=21, srv1527369217=32, srv1661116087=39, srv2055656395=63, srv569801155=81, srv511453099=79, srv32424961=72, srv1009636015=1, srv1213775560=15, srv2090428=65, srv798869826=93, srv981218582=98, srv1747543675=46, srv724187010=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:26,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:26,815 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:26,816 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:26,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 00:53:26,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 00:53:26,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 00:53:26,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 00:53:26,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 00:53:26,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 00:53:26,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 00:53:26,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 00:53:26,968 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:26,968 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 00:53:26,968 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 00:53:27,315 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 472 ms to try 5000 different iterations.  Found a solution that moves 4685 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18215606263915682. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039489193882257294, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0571567243074496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5205555555555555, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039489193882257294, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17741666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24383333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:27,339 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv790092678:80 , srv1287465525:82 , srv2037840305:82 , srv602926451:83 , srv1064254193:84 , srv1171403200:84 , srv1452266744:84 , srv1566762735:84 , srv16779267:84 , srv2014428848:84 , srv327447716:84 , srv416815301:84 , srv42171959:84 , srv798869826:84 , srv918645950:84 , srv1009871517:85 , srv1095153082:85 , srv1108457568:85 , srv1237337226:85 , srv143450436:85 , srv1458932542:85 , srv1467545817:85 , srv1525979068:85 , srv1592536003:85 , srv1661116087:85 , srv1840713835:85 , srv1876200360:85 , srv1951705434:85 , srv199306962:85 , srv2019672894:85 , srv2073235983:85 , srv2102541958:85 , srv291159477:85 , srv575909051:85 , srv666569593:85 , srv724187010:85 , srv775106451:85 , srv1009636015:86 , srv1029023651:86 , srv1086231570:86 , srv1629301089:86 , srv1632865003:86 , srv1634879736:86 , srv1683164339:86 , srv171257298:86 , srv1744734291:86 , srv1769266985:86 , srv183758406:86 , srv188778820:86 , srv511453099:86 , srv573429280:86 , srv981218582:86 , srv1038917213:87 , srv1146345105:87 , srv1219016369:87 , srv1255693594:87 , srv1273737503:87 , srv1364757860:87 , srv1395643280:87 , srv1408742299:87 , srv1462049003:87 , srv1542231432:87 , srv1826032875:87 , srv1922897018:87 , srv2015009032:87 , srv2029221234:87 , srv251234794:87 , srv268281654:87 , srv349918269:87 , srv604146866:87 , srv636415128:87 , srv806253023:87 , srv974687775:87 , srv100469462:88 , srv1212679138:88 , srv1213775560:88 , srv1829160154:88 , srv191162910:88 , srv2055656395:88 , srv32424961:88 , srv416781929:88 , srv420751634:88 , srv569801155:88 , srv714331115:88 , srv743148795:88 , srv93610488:88 , srv103373762:89 , srv1512119709:89 , srv1527369217:89 , srv1701527495:89 , srv1707946553:89 , srv1747543675:89 , srv211401934:89 , srv558798104:89 , srv984589955:89 , srv2090428:90 , srv31559955:92 , srv1030531371:93 , srv1194811758:101 , srv1472164647:441 }
2023-10-03 00:53:27,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:27,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv790092678=92, srv1632865003=37, srv349918269=74, srv1009871517=2, srv191162910=54, srv2029221234=61, srv1826032875=48, srv188778820=53, srv1146345105=11, srv171257298=44, srv268281654=69, srv199306962=57, srv1095153082=9, srv183758406=50, srv291159477=70, srv1287465525=20, srv602926451=84, srv1452266744=25, srv1395643280=22, srv1467545817=28, srv143450436=24, srv1525979068=31, srv573429280=82, srv1829160154=49, srv93610488=96, srv1194811758=13, srv211401934=67, srv743148795=90, srv1629301089=36, srv1472164647=29, srv416781929=75, srv918645950=95, srv1634879736=38, srv1255693594=18, srv1683164339=41, srv1212679138=14, srv1707946553=43, srv1542231432=33, srv2014428848=58, srv714331115=88, srv806253023=94, srv1108457568=10, srv1876200360=52, srv1951705434=56, srv327447716=73, srv984589955=99, srv31559955=71, srv1237337226=17, srv1769266985=47, srv775106451=91, srv1064254193=7, srv416815301=76, srv16779267=40, srv1566762735=34, srv100469462=0, srv2037840305=62, srv1592536003=35, srv2102541958=66, srv974687775=97, srv42171959=78, srv1922897018=55, srv2019672894=60, srv251234794=68, srv1512119709=30, srv420751634=77, srv558798104=80, srv1219016369=16, srv1408742299=23, srv1458932542=26, srv1744734291=45, srv1030531371=4, srv666569593=87, srv103373762=5, srv2015009032=59, srv2073235983=64, srv604146866=85, srv1171403200=12, srv1273737503=19, srv575909051=83, srv636415128=86, srv1701527495=42, srv1840713835=51, srv1029023651=3, srv1462049003=27, srv1086231570=8, srv1038917213=6, srv1364757860=21, srv1527369217=32, srv1661116087=39, srv2055656395=63, srv569801155=81, srv511453099=79, srv32424961=72, srv1009636015=1, srv1213775560=15, srv2090428=65, srv798869826=93, srv981218582=98, srv1747543675=46, srv724187010=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:27,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:27,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:27,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:27,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 00:53:27,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 00:53:27,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 00:53:27,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 00:53:27,374 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 00:53:27,375 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 00:53:27,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 00:53:27,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 00:53:27,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 00:53:27,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 00:53:27,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 00:53:27,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 00:53:27,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 00:53:27,379 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 00:53:27,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039489193882257294, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0571567243074496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039489193882257294, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17741666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24383333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:27,538 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 00:53:27,538 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18212325756799996, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039489193882257294, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0571567243074496, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.039489193882257294, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17741666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24383333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 00:53:27,908 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 508 ms to try 5000 different iterations.  Found a solution that moves 2638 regions; Going from a computed imbalance of 0.18212325756799996 to a new imbalance of 0.015478902553400821. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021376149514140266); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03839074897918691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2931111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021376149514140266); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043666666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:27,926 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1194811758:74 , srv31559955:79 , srv1030531371:80 , srv2090428:81 , srv420751634:82 , srv100469462:83 , srv103373762:83 , srv1212679138:83 , srv1527369217:83 , srv1747543675:83 , srv191162910:83 , srv211401934:83 , srv558798104:83 , srv743148795:83 , srv1213775560:84 , srv1273737503:84 , srv1408742299:84 , srv1462049003:84 , srv1701527495:84 , srv1707946553:84 , srv1829160154:84 , srv1922897018:84 , srv2015009032:84 , srv2055656395:84 , srv268281654:84 , srv32424961:84 , srv416781929:84 , srv569801155:84 , srv714331115:84 , srv974687775:84 , srv984589955:84 , srv1029023651:85 , srv1146345105:85 , srv1255693594:85 , srv1395643280:85 , srv1467545817:85 , srv1512119709:85 , srv1525979068:85 , srv1542231432:85 , srv1592536003:85 , srv1629301089:85 , srv1632865003:85 , srv1683164339:85 , srv171257298:85 , srv1769266985:85 , srv1876200360:85 , srv188778820:85 , srv2019672894:85 , srv511453099:85 , srv575909051:85 , srv93610488:85 , srv1038917213:86 , srv1086231570:86 , srv1095153082:86 , srv1219016369:86 , srv1661116087:86 , srv1951705434:86 , srv2073235983:86 , srv251234794:86 , srv291159477:86 , srv349918269:86 , srv573429280:86 , srv666569593:86 , srv724187010:86 , srv981218582:86 , srv1009636015:87 , srv1064254193:87 , srv1108457568:87 , srv1364757860:87 , srv143450436:87 , srv1744734291:87 , srv1826032875:87 , srv183758406:87 , srv199306962:87 , srv2014428848:87 , srv2029221234:87 , srv327447716:87 , srv604146866:87 , srv1171403200:88 , srv16779267:88 , srv1840713835:88 , srv416815301:88 , srv42171959:88 , srv775106451:88 , srv806253023:88 , srv1009871517:89 , srv1287465525:89 , srv1458932542:89 , srv1634879736:89 , srv2102541958:89 , srv602926451:89 , srv798869826:89 , srv918645950:89 , srv1237337226:90 , srv1452266744:90 , srv1566762735:90 , srv636415128:90 , srv2037840305:91 , srv790092678:92 , srv1472164647:523 }
2023-10-03 00:53:27,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:27,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv790092678=92, srv1632865003=37, srv349918269=74, srv1009871517=2, srv191162910=54, srv2029221234=61, srv1826032875=48, srv188778820=53, srv1146345105=11, srv171257298=44, srv268281654=69, srv199306962=57, srv1095153082=9, srv183758406=50, srv291159477=70, srv1287465525=20, srv602926451=84, srv1452266744=25, srv1395643280=22, srv1467545817=28, srv143450436=24, srv1525979068=31, srv573429280=82, srv1829160154=49, srv93610488=96, srv1194811758=13, srv211401934=67, srv743148795=90, srv1629301089=36, srv1472164647=29, srv416781929=75, srv918645950=95, srv1634879736=38, srv1255693594=18, srv1683164339=41, srv1212679138=14, srv1707946553=43, srv1542231432=33, srv2014428848=58, srv714331115=88, srv806253023=94, srv1108457568=10, srv1876200360=52, srv1951705434=56, srv327447716=73, srv984589955=99, srv31559955=71, srv1237337226=17, srv1769266985=47, srv775106451=91, srv1064254193=7, srv416815301=76, srv16779267=40, srv1566762735=34, srv100469462=0, srv2037840305=62, srv1592536003=35, srv2102541958=66, srv974687775=97, srv42171959=78, srv1922897018=55, srv2019672894=60, srv251234794=68, srv1512119709=30, srv420751634=77, srv558798104=80, srv1219016369=16, srv1408742299=23, srv1458932542=26, srv1744734291=45, srv1030531371=4, srv666569593=87, srv103373762=5, srv2015009032=59, srv2073235983=64, srv604146866=85, srv1171403200=12, srv1273737503=19, srv575909051=83, srv636415128=86, srv1701527495=42, srv1840713835=51, srv1029023651=3, srv1462049003=27, srv1086231570=8, srv1038917213=6, srv1364757860=21, srv1527369217=32, srv1661116087=39, srv2055656395=63, srv569801155=81, srv511453099=79, srv32424961=72, srv1009636015=1, srv1213775560=15, srv2090428=65, srv798869826=93, srv981218582=98, srv1747543675=46, srv724187010=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:27,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:27,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:27,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:27,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:27,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:27,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:27,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:27,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 00:53:27,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 00:53:27,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 00:53:27,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 00:53:27,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 00:53:27,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 00:53:27,983 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 00:53:27,984 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 00:53:27,985 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 00:53:28,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021376149514140266); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03839074897918691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021376149514140266); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043666666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:28,178 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 00:53:28,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015460430882597884, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.021376149514140266); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03839074897918691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.021376149514140266); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0125); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043666666666666666, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 00:53:28,582 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 573 ms to try 5000 different iterations.  Found a solution that moves 1039 regions; Going from a computed imbalance of 0.015460430882597884 to a new imbalance of 6.466777497279895E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0056567660698823964); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012659753562742059); MoveCostFunction : (multiplier=7.0, imbalance=0.11544444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0056567660698823964); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:28,585 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv636415128:81 , srv1634879736:82 , srv1237337226:83 , srv1452266744:83 , srv1512119709:83 , srv2029221234:83 , srv2102541958:83 , srv1009636015:84 , srv1009871517:84 , srv103373762:84 , srv1038917213:84 , srv1194811758:84 , srv1364757860:84 , srv1458932542:84 , srv1566762735:84 , srv1707946553:84 , srv1826032875:84 , srv1840713835:84 , srv2037840305:84 , srv604146866:84 , srv798869826:84 , srv806253023:84 , srv918645950:84 , srv1029023651:85 , srv1030531371:85 , srv1146345105:85 , srv1213775560:85 , srv1219016369:85 , srv1255693594:85 , srv1395643280:85 , srv143450436:85 , srv1527369217:85 , srv1542231432:85 , srv16779267:85 , srv1701527495:85 , srv1744734291:85 , srv183758406:85 , srv199306962:85 , srv2055656395:85 , srv251234794:85 , srv32424961:85 , srv416781929:85 , srv42171959:85 , srv558798104:85 , srv714331115:85 , srv775106451:85 , srv984589955:85 , srv1095153082:86 , srv1108457568:86 , srv1171403200:86 , srv1212679138:86 , srv1273737503:86 , srv1287465525:86 , srv1408742299:86 , srv1462049003:86 , srv1632865003:86 , srv1683164339:86 , srv171257298:86 , srv1747543675:86 , srv1769266985:86 , srv1829160154:86 , srv188778820:86 , srv191162910:86 , srv1922897018:86 , srv2014428848:86 , srv2090428:86 , srv211401934:86 , srv268281654:86 , srv291159477:86 , srv31559955:86 , srv327447716:86 , srv349918269:86 , srv416815301:86 , srv511453099:86 , srv569801155:86 , srv573429280:86 , srv602926451:86 , srv724187010:86 , srv790092678:86 , srv93610488:86 , srv981218582:86 , srv100469462:87 , srv1064254193:87 , srv1086231570:87 , srv1525979068:87 , srv1592536003:87 , srv1629301089:87 , srv1661116087:87 , srv1876200360:87 , srv1951705434:87 , srv2015009032:87 , srv2073235983:87 , srv420751634:87 , srv743148795:87 , srv974687775:87 , srv1467545817:88 , srv2019672894:88 , srv575909051:88 , srv666569593:88 , srv1472164647:544 }
2023-10-03 00:53:28,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:28,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv790092678=92, srv1632865003=37, srv349918269=74, srv1009871517=2, srv191162910=54, srv2029221234=61, srv1826032875=48, srv188778820=53, srv1146345105=11, srv171257298=44, srv268281654=69, srv199306962=57, srv1095153082=9, srv183758406=50, srv291159477=70, srv1287465525=20, srv602926451=84, srv1452266744=25, srv1395643280=22, srv1467545817=28, srv143450436=24, srv1525979068=31, srv573429280=82, srv1829160154=49, srv93610488=96, srv1194811758=13, srv211401934=67, srv743148795=90, srv1629301089=36, srv1472164647=29, srv416781929=75, srv918645950=95, srv1634879736=38, srv1255693594=18, srv1683164339=41, srv1212679138=14, srv1707946553=43, srv1542231432=33, srv2014428848=58, srv714331115=88, srv806253023=94, srv1108457568=10, srv1876200360=52, srv1951705434=56, srv327447716=73, srv984589955=99, srv31559955=71, srv1237337226=17, srv1769266985=47, srv775106451=91, srv1064254193=7, srv416815301=76, srv16779267=40, srv1566762735=34, srv100469462=0, srv2037840305=62, srv1592536003=35, srv2102541958=66, srv974687775=97, srv42171959=78, srv1922897018=55, srv2019672894=60, srv251234794=68, srv1512119709=30, srv420751634=77, srv558798104=80, srv1219016369=16, srv1408742299=23, srv1458932542=26, srv1744734291=45, srv1030531371=4, srv666569593=87, srv103373762=5, srv2015009032=59, srv2073235983=64, srv604146866=85, srv1171403200=12, srv1273737503=19, srv575909051=83, srv636415128=86, srv1701527495=42, srv1840713835=51, srv1029023651=3, srv1462049003=27, srv1086231570=8, srv1038917213=6, srv1364757860=21, srv1527369217=32, srv1661116087=39, srv2055656395=63, srv569801155=81, srv511453099=79, srv32424961=72, srv1009636015=1, srv1213775560=15, srv2090428=65, srv798869826=93, srv981218582=98, srv1747543675=46, srv724187010=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:28,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:28,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:28,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:28,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:28,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 00:53:28,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 00:53:28,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 00:53:28,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 00:53:28,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 00:53:28,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 00:53:28,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 00:53:28,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 00:53:28,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 00:53:28,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 00:53:28,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 00:53:28,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 00:53:28,776 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0056567660698823964); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012659753562742059); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0056567660698823964); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:28,776 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 00:53:28,776 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=6.39402516231306E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0056567660698823964); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.012659753562742059); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0056567660698823964); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 00:53:29,263 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 625 ms to try 5000 different iterations.  Found a solution that moves 274 regions; Going from a computed imbalance of 6.39402516231306E-4 to a new imbalance of 5.957044603389012E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.030444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:29,264 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1194811758:82 , srv1744734291:83 , srv666569593:83 , srv806253023:83 , srv93610488:83 , srv100469462:84 , srv1009636015:84 , srv1009871517:84 , srv1030531371:84 , srv1064254193:84 , srv1086231570:84 , srv1108457568:84 , srv1171403200:84 , srv1213775560:84 , srv1237337226:84 , srv1364757860:84 , srv1467545817:84 , srv1542231432:84 , srv1566762735:84 , srv1629301089:84 , srv1661116087:84 , srv1683164339:84 , srv1701527495:84 , srv1747543675:84 , srv1826032875:84 , srv1829160154:84 , srv188778820:84 , srv1951705434:84 , srv2015009032:84 , srv2019672894:84 , srv2029221234:84 , srv2073235983:84 , srv2102541958:84 , srv211401934:84 , srv268281654:84 , srv327447716:84 , srv349918269:84 , srv416815301:84 , srv569801155:84 , srv573429280:84 , srv575909051:84 , srv602926451:84 , srv604146866:84 , srv636415128:84 , srv743148795:84 , srv775106451:84 , srv790092678:84 , srv798869826:84 , srv974687775:84 , srv981218582:84 , srv1038917213:85 , srv1095153082:85 , srv1146345105:85 , srv1212679138:85 , srv1219016369:85 , srv1255693594:85 , srv1273737503:85 , srv1287465525:85 , srv1395643280:85 , srv1408742299:85 , srv143450436:85 , srv1452266744:85 , srv1458932542:85 , srv1462049003:85 , srv1512119709:85 , srv1525979068:85 , srv1527369217:85 , srv1592536003:85 , srv1632865003:85 , srv1634879736:85 , srv171257298:85 , srv1769266985:85 , srv183758406:85 , srv1840713835:85 , srv191162910:85 , srv1922897018:85 , srv199306962:85 , srv2014428848:85 , srv2037840305:85 , srv2090428:85 , srv251234794:85 , srv291159477:85 , srv31559955:85 , srv32424961:85 , srv416781929:85 , srv420751634:85 , srv42171959:85 , srv511453099:85 , srv558798104:85 , srv724187010:85 , srv918645950:85 , srv984589955:85 , srv1029023651:86 , srv103373762:86 , srv16779267:86 , srv1707946553:86 , srv1876200360:86 , srv2055656395:86 , srv714331115:86 , srv1472164647:634 }
2023-10-03 00:53:29,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:29,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv790092678=92, srv1632865003=37, srv349918269=74, srv1009871517=2, srv191162910=54, srv2029221234=61, srv1826032875=48, srv188778820=53, srv1146345105=11, srv171257298=44, srv268281654=69, srv199306962=57, srv1095153082=9, srv183758406=50, srv291159477=70, srv1287465525=20, srv602926451=84, srv1452266744=25, srv1395643280=22, srv1467545817=28, srv143450436=24, srv1525979068=31, srv573429280=82, srv1829160154=49, srv93610488=96, srv1194811758=13, srv211401934=67, srv743148795=90, srv1629301089=36, srv1472164647=29, srv416781929=75, srv918645950=95, srv1634879736=38, srv1255693594=18, srv1683164339=41, srv1212679138=14, srv1707946553=43, srv1542231432=33, srv2014428848=58, srv714331115=88, srv806253023=94, srv1108457568=10, srv1876200360=52, srv1951705434=56, srv327447716=73, srv984589955=99, srv31559955=71, srv1237337226=17, srv1769266985=47, srv775106451=91, srv1064254193=7, srv416815301=76, srv16779267=40, srv1566762735=34, srv100469462=0, srv2037840305=62, srv1592536003=35, srv2102541958=66, srv974687775=97, srv42171959=78, srv1922897018=55, srv2019672894=60, srv251234794=68, srv1512119709=30, srv420751634=77, srv558798104=80, srv1219016369=16, srv1408742299=23, srv1458932542=26, srv1744734291=45, srv1030531371=4, srv666569593=87, srv103373762=5, srv2015009032=59, srv2073235983=64, srv604146866=85, srv1171403200=12, srv1273737503=19, srv575909051=83, srv636415128=86, srv1701527495=42, srv1840713835=51, srv1029023651=3, srv1462049003=27, srv1086231570=8, srv1038917213=6, srv1364757860=21, srv1527369217=32, srv1661116087=39, srv2055656395=63, srv569801155=81, srv511453099=79, srv32424961=72, srv1009636015=1, srv1213775560=15, srv2090428=65, srv798869826=93, srv981218582=98, srv1747543675=46, srv724187010=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:29,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:29,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:29,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 00:53:29,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 00:53:29,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 00:53:29,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 00:53:29,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 00:53:29,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 00:53:29,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 00:53:29,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 00:53:29,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 00:53:29,431 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:29,431 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-03 00:53:29,432 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.7651857027065025E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=5.470732932414788E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022222222222222222); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=5.470732932414788E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=5.0E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-03 00:53:29,744 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 434 ms to try 5000 different iterations.  Found a solution that moves 47 regions; Going from a computed imbalance of 5.7651857027065025E-5 to a new imbalance of 6.727086706287142E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.005222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:29,745 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1086231570:83 , srv1219016369:83 , srv1458932542:83 , srv16779267:83 , srv1707946553:83 , srv183758406:83 , srv1876200360:83 , srv2055656395:83 , srv251234794:83 , srv349918269:83 , srv714331115:83 , srv984589955:83 , srv100469462:84 , srv1009871517:84 , srv1029023651:84 , srv1030531371:84 , srv103373762:84 , srv1038917213:84 , srv1064254193:84 , srv1095153082:84 , srv1108457568:84 , srv1146345105:84 , srv1171403200:84 , srv1212679138:84 , srv1237337226:84 , srv1255693594:84 , srv1273737503:84 , srv1287465525:84 , srv1364757860:84 , srv1395643280:84 , srv1408742299:84 , srv143450436:84 , srv1452266744:84 , srv1462049003:84 , srv1467545817:84 , srv1512119709:84 , srv1525979068:84 , srv1527369217:84 , srv1566762735:84 , srv1592536003:84 , srv1629301089:84 , srv1632865003:84 , srv1634879736:84 , srv1661116087:84 , srv1701527495:84 , srv171257298:84 , srv1747543675:84 , srv1769266985:84 , srv1826032875:84 , srv1829160154:84 , srv1840713835:84 , srv191162910:84 , srv1922897018:84 , srv1951705434:84 , srv199306962:84 , srv2014428848:84 , srv2015009032:84 , srv2019672894:84 , srv2037840305:84 , srv2073235983:84 , srv2090428:84 , srv211401934:84 , srv291159477:84 , srv31559955:84 , srv32424961:84 , srv416781929:84 , srv416815301:84 , srv420751634:84 , srv42171959:84 , srv511453099:84 , srv558798104:84 , srv569801155:84 , srv573429280:84 , srv575909051:84 , srv602926451:84 , srv604146866:84 , srv636415128:84 , srv666569593:84 , srv724187010:84 , srv743148795:84 , srv775106451:84 , srv790092678:84 , srv806253023:84 , srv918645950:84 , srv93610488:84 , srv974687775:84 , srv981218582:84 , srv1009636015:85 , srv1194811758:85 , srv1213775560:85 , srv1542231432:85 , srv1683164339:85 , srv1744734291:85 , srv188778820:85 , srv2029221234:85 , srv2102541958:85 , srv268281654:85 , srv327447716:85 , srv798869826:85 , srv1472164647:684 }
2023-10-03 00:53:29,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:29,751 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv790092678=92, srv1632865003=37, srv349918269=74, srv1009871517=2, srv191162910=54, srv2029221234=61, srv1826032875=48, srv188778820=53, srv1146345105=11, srv171257298=44, srv268281654=69, srv199306962=57, srv1095153082=9, srv183758406=50, srv291159477=70, srv1287465525=20, srv602926451=84, srv1452266744=25, srv1395643280=22, srv1467545817=28, srv143450436=24, srv1525979068=31, srv573429280=82, srv1829160154=49, srv93610488=96, srv1194811758=13, srv211401934=67, srv743148795=90, srv1629301089=36, srv1472164647=29, srv416781929=75, srv918645950=95, srv1634879736=38, srv1255693594=18, srv1683164339=41, srv1212679138=14, srv1707946553=43, srv1542231432=33, srv2014428848=58, srv714331115=88, srv806253023=94, srv1108457568=10, srv1876200360=52, srv1951705434=56, srv327447716=73, srv984589955=99, srv31559955=71, srv1237337226=17, srv1769266985=47, srv775106451=91, srv1064254193=7, srv416815301=76, srv16779267=40, srv1566762735=34, srv100469462=0, srv2037840305=62, srv1592536003=35, srv2102541958=66, srv974687775=97, srv42171959=78, srv1922897018=55, srv2019672894=60, srv251234794=68, srv1512119709=30, srv420751634=77, srv558798104=80, srv1219016369=16, srv1408742299=23, srv1458932542=26, srv1744734291=45, srv1030531371=4, srv666569593=87, srv103373762=5, srv2015009032=59, srv2073235983=64, srv604146866=85, srv1171403200=12, srv1273737503=19, srv575909051=83, srv636415128=86, srv1701527495=42, srv1840713835=51, srv1029023651=3, srv1462049003=27, srv1086231570=8, srv1038917213=6, srv1364757860=21, srv1527369217=32, srv1661116087=39, srv2055656395=63, srv569801155=81, srv511453099=79, srv32424961=72, srv1009636015=1, srv1213775560=15, srv2090428=65, srv798869826=93, srv981218582=98, srv1747543675=46, srv724187010=89} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:29,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:29,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:29,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:29,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:29,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:29,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:29,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-03 00:53:29,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-03 00:53:29,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-03 00:53:29,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-03 00:53:29,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-03 00:53:29,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-03 00:53:29,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-03 00:53:29,930 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:29,930 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1086231570:83 , srv1219016369:83 , srv1458932542:83 , srv16779267:83 , srv1707946553:83 , srv183758406:83 , srv1876200360:83 , srv2055656395:83 , srv251234794:83 , srv349918269:83 , srv714331115:83 , srv984589955:83 , srv100469462:84 , srv1009871517:84 , srv1029023651:84 , srv1030531371:84 , srv103373762:84 , srv1038917213:84 , srv1064254193:84 , srv1095153082:84 , srv1108457568:84 , srv1146345105:84 , srv1171403200:84 , srv1212679138:84 , srv1237337226:84 , srv1255693594:84 , srv1273737503:84 , srv1287465525:84 , srv1364757860:84 , srv1395643280:84 , srv1408742299:84 , srv143450436:84 , srv1452266744:84 , srv1462049003:84 , srv1467545817:84 , srv1512119709:84 , srv1525979068:84 , srv1527369217:84 , srv1566762735:84 , srv1592536003:84 , srv1629301089:84 , srv1632865003:84 , srv1634879736:84 , srv1661116087:84 , srv1701527495:84 , srv171257298:84 , srv1747543675:84 , srv1769266985:84 , srv1826032875:84 , srv1829160154:84 , srv1840713835:84 , srv191162910:84 , srv1922897018:84 , srv1951705434:84 , srv199306962:84 , srv2014428848:84 , srv2015009032:84 , srv2019672894:84 , srv2037840305:84 , srv2073235983:84 , srv2090428:84 , srv211401934:84 , srv291159477:84 , srv31559955:84 , srv32424961:84 , srv416781929:84 , srv416815301:84 , srv420751634:84 , srv42171959:84 , srv511453099:84 , srv558798104:84 , srv569801155:84 , srv573429280:84 , srv575909051:84 , srv602926451:84 , srv604146866:84 , srv636415128:84 , srv666569593:84 , srv724187010:84 , srv743148795:84 , srv775106451:84 , srv790092678:84 , srv806253023:84 , srv918645950:84 , srv93610488:84 , srv974687775:84 , srv981218582:84 , srv1009636015:85 , srv1194811758:85 , srv1213775560:85 , srv1542231432:85 , srv1683164339:85 , srv1744734291:85 , srv188778820:85 , srv2029221234:85 , srv2102541958:85 , srv268281654:85 , srv327447716:85 , srv798869826:85 , srv1472164647:684 }
2023-10-03 00:53:29,964 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=303 (was 277) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=9480 (was 9533)
2023-10-03 00:53:29,976 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=303, ProcessCount=171, AvailableMemoryMB=9477
2023-10-03 00:53:29,978 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-03 00:53:29,978 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-03 00:53:29,992 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1119514173:84 , srv1227921484:84 , srv1228478493:84 , srv1376708916:84 , srv1469599755:84 , srv14916681:84 , srv1494821401:84 , srv1555765458:84 , srv1634396038:84 , srv16381175:84 , srv1663630793:84 , srv1837882082:84 , srv1897968956:84 , srv2112622191:84 , srv2137684821:84 , srv271401511:84 , srv36232233:84 , srv37313621:84 , srv389197944:84 , srv420650254:84 , srv577373375:84 , srv610638088:84 , srv654186268:84 , srv672416767:84 , srv802004923:84 , srv821076792:84 , srv900291464:84 , srv970902347:84 , srv99637445:84 , srv1932965082:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-03 00:53:29,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:53:29,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1634396038=8, srv14916681=5, srv99637445=29, srv577373375=21, srv2112622191=14, srv420650254=20, srv1555765458=7, srv1837882082=11, srv821076792=26, srv1227921484=1, srv970902347=28, srv1119514173=0, srv36232233=17, srv1897968956=12, srv900291464=27, srv802004923=25, srv16381175=9, srv1932965082=13, srv2137684821=15, srv389197944=19, srv1376708916=3, srv610638088=22, srv271401511=16, srv1228478493=2, srv1494821401=6, srv1469599755=4, srv672416767=24, srv37313621=18, srv1663630793=10, srv654186268=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:53:30,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:53:30,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:53:30,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-03 00:53:30,029 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0596574867286408, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:53:30,030 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912381022056966, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0596574867286408, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-03 00:55:30,011 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5535554 different iterations.  Found a solution that moves 1860 regions; Going from a computed imbalance of 0.9912381022056966 to a new imbalance of 4.341332789166274E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6888888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:55:30,018 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1119514173:90 , srv1227921484:90 , srv1228478493:90 , srv1376708916:90 , srv1469599755:90 , srv14916681:90 , srv1494821401:90 , srv1555765458:90 , srv1634396038:90 , srv16381175:90 , srv1663630793:90 , srv1837882082:90 , srv1897968956:90 , srv1932965082:90 , srv2112622191:90 , srv2137684821:90 , srv271401511:90 , srv36232233:90 , srv37313621:90 , srv389197944:90 , srv420650254:90 , srv577373375:90 , srv610638088:90 , srv654186268:90 , srv672416767:90 , srv802004923:90 , srv821076792:90 , srv900291464:90 , srv970902347:90 , srv99637445:90 }
2023-10-03 00:55:30,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-03 00:55:30,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1634396038=8, srv14916681=5, srv99637445=29, srv577373375=21, srv2112622191=14, srv420650254=20, srv1555765458=7, srv1837882082=11, srv821076792=26, srv1227921484=1, srv970902347=28, srv1119514173=0, srv36232233=17, srv1897968956=12, srv900291464=27, srv802004923=25, srv16381175=9, srv1932965082=13, srv2137684821=15, srv389197944=19, srv1376708916=3, srv610638088=22, srv271401511=16, srv1228478493=2, srv1494821401=6, srv1469599755=4, srv672416767=24, srv37313621=18, srv1663630793=10, srv654186268=23} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-03 00:55:30,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-03 00:55:30,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-03 00:55:30,037 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-03 00:55:30,037 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1119514173:90 , srv1227921484:90 , srv1228478493:90 , srv1376708916:90 , srv1469599755:90 , srv14916681:90 , srv1494821401:90 , srv1555765458:90 , srv1634396038:90 , srv16381175:90 , srv1663630793:90 , srv1837882082:90 , srv1897968956:90 , srv1932965082:90 , srv2112622191:90 , srv2137684821:90 , srv271401511:90 , srv36232233:90 , srv37313621:90 , srv389197944:90 , srv420650254:90 , srv577373375:90 , srv610638088:90 , srv654186268:90 , srv672416767:90 , srv802004923:90 , srv821076792:90 , srv900291464:90 , srv970902347:90 , srv99637445:90 }
2023-10-03 00:55:30,050 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 303), ProcessCount=172 (was 171) - ProcessCount LEAK? -, AvailableMemoryMB=8195 (was 9477)