2023-09-19 04:53:18,534 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-19 04:53:18,870 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-19 04:53:18,988 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-19 04:53:19,050 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-19 04:53:19,060 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-19 04:53:19,078 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=367, ProcessCount=170, AvailableMemoryMB=10791
2023-09-19 04:53:19,080 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-19 04:53:19,080 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-19 04:53:19,230 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1027154246:84 , srv1038324665:84 , srv1082755414:84 , srv1156690619:84 , srv1192040911:84 , srv1209459637:84 , srv1217398726:84 , srv121846343:84 , srv1220159557:84 , srv1244013594:84 , srv1256096544:84 , srv1282157263:84 , srv1313234554:84 , srv1324117890:84 , srv1328640653:84 , srv1371666944:84 , srv1376520336:84 , srv1378375528:84 , srv1430803481:84 , srv1435728896:84 , srv1441778036:84 , srv1456972594:84 , srv1467099224:84 , srv147975630:84 , srv1492029926:84 , srv154419449:84 , srv1568243477:84 , srv158724490:84 , srv1597357863:84 , srv1607515749:84 , srv1607555193:84 , srv1616915136:84 , srv1626766036:84 , srv1651888318:84 , srv1674573702:84 , srv1728317916:84 , srv1751937367:84 , srv1790566797:84 , srv1809601176:84 , srv1812288523:84 , srv185110058:84 , srv1883987140:84 , srv1897129028:84 , srv190042652:84 , srv1906546949:84 , srv1915502160:84 , srv1920635465:84 , srv193235380:84 , srv1961370827:84 , srv2035840255:84 , srv2082513551:84 , srv2090166666:84 , srv2095201424:84 , srv2098504385:84 , srv211442704:84 , srv2123302318:84 , srv220170697:84 , srv221927050:84 , srv2467381:84 , srv273395332:84 , srv284676685:84 , srv289964572:84 , srv317807086:84 , srv350014950:84 , srv350692199:84 , srv366880201:84 , srv413024907:84 , srv436293677:84 , srv445983686:84 , srv460684685:84 , srv466590853:84 , srv473348208:84 , srv473375391:84 , srv490025992:84 , srv50803142:84 , srv515342143:84 , srv518623878:84 , srv553630384:84 , srv55710863:84 , srv558232378:84 , srv558433630:84 , srv566578945:84 , srv567949125:84 , srv597945969:84 , srv61790466:84 , srv676821733:84 , srv682425885:84 , srv73159367:84 , srv745874507:84 , srv752294556:84 , srv767734812:84 , srv798974490:84 , srv812495635:84 , srv889862470:84 , srv894219606:84 , srv953207268:84 , srv960762199:84 , srv97610462:84 , srv984416122:84 , srv1700042534:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-19 04:53:19,243 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:53:19,251 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1790566797=38, srv466590853=71, srv147975630=23, srv2082513551=51, srv960762199=97, srv350692199=65, srv413024907=67, srv953207268=96, srv1651888318=33, srv97610462=98, srv1674573702=34, srv190042652=44, srv2098504385=54, srv1920635465=47, srv1192040911=4, srv2095201424=53, srv220170697=57, srv1456972594=21, srv984416122=99, srv445983686=69, srv1700042534=35, srv1435728896=19, srv1027154246=0, srv158724490=27, srv1492029926=24, srv1156690619=3, srv1378375528=17, srv121846343=7, srv2467381=59, srv284676685=61, srv1616915136=31, srv1897129028=43, srv1217398726=6, srv350014950=64, srv1728317916=36, srv515342143=76, srv767734812=91, srv1371666944=15, srv1915502160=46, srv682425885=87, srv1906546949=45, srv1244013594=9, srv473375391=73, srv193235380=48, srv1568243477=26, srv1256096544=10, srv1282157263=11, srv889862470=94, srv1220159557=8, srv1441778036=20, srv211442704=55, srv1883987140=42, srv1607555193=30, srv460684685=70, srv1328640653=14, srv273395332=60, srv553630384=78, srv1467099224=22, srv566578945=82, srv73159367=88, srv2123302318=56, srv1430803481=18, srv289964572=62, srv798974490=92, srv1809601176=39, srv1607515749=29, srv317807086=63, srv1751937367=37, srv366880201=66, srv50803142=75, srv61790466=85, srv597945969=84, srv1626766036=32, srv1038324665=1, srv490025992=74, srv558433630=81, srv1324117890=13, srv812495635=93, srv567949125=83, srv154419449=25, srv436293677=68, srv518623878=77, srv55710863=79, srv1313234554=12, srv221927050=58, srv745874507=89, srv2035840255=50, srv894219606=95, srv1209459637=5, srv1961370827=49, srv1082755414=2, srv1376520336=16, srv185110058=41, srv752294556=90, srv1597357863=28, srv1812288523=40, srv676821733=86, srv558232378=80, srv2090166666=52, srv473348208=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:53:19,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:53:19,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:53:19,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:53:19,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-19 04:53:19,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-19 04:53:19,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-19 04:53:19,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-19 04:53:19,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-19 04:53:19,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-19 04:53:19,333 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-19 04:53:19,334 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-19 04:53:19,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-19 04:53:19,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-19 04:53:19,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-19 04:53:19,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-19 04:53:19,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-19 04:53:19,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-19 04:53:19,336 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-19 04:53:19,337 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-19 04:53:19,508 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:19,509 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-19 04:53:19,509 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-19 04:53:19,875 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 511 ms to try 5000 different iterations.  Found a solution that moves 4692 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17939086088451459. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03892494720807615, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053448948759136417, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5213333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03892494720807615, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17458333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24166666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:19,893 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1441778036:81 , srv1607515749:82 , srv1915502160:82 , srv2098504385:82 , srv2123302318:82 , srv676821733:82 , srv1027154246:83 , srv1192040911:83 , srv1607555193:83 , srv273395332:83 , srv50803142:83 , srv597945969:83 , srv1674573702:84 , srv1751937367:84 , srv1809601176:84 , srv220170697:84 , srv221927050:84 , srv289964572:84 , srv558232378:84 , srv767734812:84 , srv1038324665:85 , srv1082755414:85 , srv147975630:85 , srv1492029926:85 , srv1616915136:85 , srv185110058:85 , srv350014950:85 , srv350692199:85 , srv553630384:85 , srv566578945:85 , srv61790466:85 , srv889862470:85 , srv1328640653:86 , srv1597357863:86 , srv1651888318:86 , srv1728317916:86 , srv1906546949:86 , srv1920635465:86 , srv1961370827:86 , srv2035840255:86 , srv284676685:86 , srv436293677:86 , srv473348208:86 , srv490025992:86 , srv73159367:86 , srv960762199:86 , srv97610462:86 , srv1156690619:87 , srv1209459637:87 , srv1217398726:87 , srv121846343:87 , srv1220159557:87 , srv1256096544:87 , srv1371666944:87 , srv1376520336:87 , srv154419449:87 , srv158724490:87 , srv1626766036:87 , srv211442704:87 , srv466590853:87 , srv473375391:87 , srv798974490:87 , srv953207268:87 , srv984416122:87 , srv1313234554:88 , srv1324117890:88 , srv1378375528:88 , srv1435728896:88 , srv1456972594:88 , srv1467099224:88 , srv1812288523:88 , srv1883987140:88 , srv190042652:88 , srv2082513551:88 , srv317807086:88 , srv366880201:88 , srv515342143:88 , srv518623878:88 , srv55710863:88 , srv558433630:88 , srv745874507:88 , srv894219606:88 , srv1282157263:89 , srv2090166666:89 , srv2095201424:89 , srv2467381:89 , srv413024907:89 , srv445983686:89 , srv460684685:89 , srv812495635:89 , srv1430803481:90 , srv1568243477:90 , srv193235380:90 , srv567949125:90 , srv682425885:90 , srv1790566797:91 , srv1897129028:91 , srv752294556:91 , srv1244013594:95 , srv1700042534:436 }
2023-09-19 04:53:19,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:53:19,901 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1790566797=38, srv466590853=71, srv147975630=23, srv2082513551=51, srv960762199=97, srv350692199=65, srv413024907=67, srv953207268=96, srv1651888318=33, srv97610462=98, srv1674573702=34, srv190042652=44, srv2098504385=54, srv1920635465=47, srv1192040911=4, srv2095201424=53, srv220170697=57, srv1456972594=21, srv984416122=99, srv445983686=69, srv1700042534=35, srv1435728896=19, srv1027154246=0, srv158724490=27, srv1492029926=24, srv1156690619=3, srv1378375528=17, srv121846343=7, srv2467381=59, srv284676685=61, srv1616915136=31, srv1897129028=43, srv1217398726=6, srv350014950=64, srv1728317916=36, srv515342143=76, srv767734812=91, srv1371666944=15, srv1915502160=46, srv682425885=87, srv1906546949=45, srv1244013594=9, srv473375391=73, srv193235380=48, srv1568243477=26, srv1256096544=10, srv1282157263=11, srv889862470=94, srv1220159557=8, srv1441778036=20, srv211442704=55, srv1883987140=42, srv1607555193=30, srv460684685=70, srv1328640653=14, srv273395332=60, srv553630384=78, srv1467099224=22, srv566578945=82, srv73159367=88, srv2123302318=56, srv1430803481=18, srv289964572=62, srv798974490=92, srv1809601176=39, srv1607515749=29, srv317807086=63, srv1751937367=37, srv366880201=66, srv50803142=75, srv61790466=85, srv597945969=84, srv1626766036=32, srv1038324665=1, srv490025992=74, srv558433630=81, srv1324117890=13, srv812495635=93, srv567949125=83, srv154419449=25, srv436293677=68, srv518623878=77, srv55710863=79, srv1313234554=12, srv221927050=58, srv745874507=89, srv2035840255=50, srv894219606=95, srv1209459637=5, srv1961370827=49, srv1082755414=2, srv1376520336=16, srv185110058=41, srv752294556=90, srv1597357863=28, srv1812288523=40, srv676821733=86, srv558232378=80, srv2090166666=52, srv473348208=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:53:19,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:53:19,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:53:19,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:53:19,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:53:19,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-19 04:53:19,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-19 04:53:19,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-19 04:53:19,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-19 04:53:19,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-19 04:53:19,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-19 04:53:19,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-19 04:53:19,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-19 04:53:19,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-19 04:53:19,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-19 04:53:19,939 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-19 04:53:20,127 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03892494720807615, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053448948759136417, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03892494720807615, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17458333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24166666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:20,127 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-19 04:53:20,128 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1793580067983101, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03892494720807615, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.053448948759136417, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03892494720807615, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17458333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24166666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-19 04:53:20,492 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 534 ms to try 5000 different iterations.  Found a solution that moves 2645 regions; Going from a computed imbalance of 0.1793580067983101 to a new imbalance of 0.016470613333385415. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019478805618229192); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027389201342287256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29388888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019478805618229192); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04533333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:20,507 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1244013594:78 , srv1790566797:79 , srv567949125:81 , srv682425885:81 , srv752294556:81 , srv1282157263:82 , srv1378375528:82 , srv1568243477:82 , srv1897129028:82 , srv193235380:82 , srv2090166666:82 , srv460684685:82 , srv515342143:82 , srv745874507:82 , srv1430803481:83 , srv190042652:83 , srv2095201424:83 , srv1435728896:84 , srv1467099224:84 , srv1883987140:84 , srv2467381:84 , srv284676685:84 , srv466590853:84 , srv1209459637:85 , srv1217398726:85 , srv1220159557:85 , srv1256096544:85 , srv1324117890:85 , srv1328640653:85 , srv1371666944:85 , srv1456972594:85 , srv158724490:85 , srv1597357863:85 , srv1616915136:85 , srv1728317916:85 , srv1920635465:85 , srv1961370827:85 , srv2082513551:85 , srv211442704:85 , srv317807086:85 , srv518623878:85 , srv558433630:85 , srv73159367:85 , srv953207268:85 , srv984416122:85 , srv121846343:86 , srv147975630:86 , srv154419449:86 , srv1651888318:86 , srv1812288523:86 , srv185110058:86 , srv1906546949:86 , srv350014950:86 , srv413024907:86 , srv436293677:86 , srv445983686:86 , srv473375391:86 , srv490025992:86 , srv61790466:86 , srv798974490:86 , srv960762199:86 , srv97610462:86 , srv1038324665:87 , srv1082755414:87 , srv1156690619:87 , srv1192040911:87 , srv1313234554:87 , srv1492029926:87 , srv1626766036:87 , srv1674573702:87 , srv1751937367:87 , srv2035840255:87 , srv220170697:87 , srv473348208:87 , srv553630384:87 , srv55710863:87 , srv812495635:87 , srv894219606:87 , srv273395332:88 , srv366880201:88 , srv50803142:88 , srv1027154246:89 , srv1376520336:89 , srv1607515749:89 , srv1809601176:89 , srv2123302318:89 , srv221927050:89 , srv350692199:89 , srv558232378:89 , srv566578945:89 , srv597945969:89 , srv767734812:89 , srv1441778036:90 , srv1607555193:90 , srv2098504385:90 , srv289964572:90 , srv676821733:90 , srv889862470:90 , srv1915502160:91 , srv1700042534:511 }
2023-09-19 04:53:20,513 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:53:20,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1790566797=38, srv466590853=71, srv147975630=23, srv2082513551=51, srv960762199=97, srv350692199=65, srv413024907=67, srv953207268=96, srv1651888318=33, srv97610462=98, srv1674573702=34, srv190042652=44, srv2098504385=54, srv1920635465=47, srv1192040911=4, srv2095201424=53, srv220170697=57, srv1456972594=21, srv984416122=99, srv445983686=69, srv1700042534=35, srv1435728896=19, srv1027154246=0, srv158724490=27, srv1492029926=24, srv1156690619=3, srv1378375528=17, srv121846343=7, srv2467381=59, srv284676685=61, srv1616915136=31, srv1897129028=43, srv1217398726=6, srv350014950=64, srv1728317916=36, srv515342143=76, srv767734812=91, srv1371666944=15, srv1915502160=46, srv682425885=87, srv1906546949=45, srv1244013594=9, srv473375391=73, srv193235380=48, srv1568243477=26, srv1256096544=10, srv1282157263=11, srv889862470=94, srv1220159557=8, srv1441778036=20, srv211442704=55, srv1883987140=42, srv1607555193=30, srv460684685=70, srv1328640653=14, srv273395332=60, srv553630384=78, srv1467099224=22, srv566578945=82, srv73159367=88, srv2123302318=56, srv1430803481=18, srv289964572=62, srv798974490=92, srv1809601176=39, srv1607515749=29, srv317807086=63, srv1751937367=37, srv366880201=66, srv50803142=75, srv61790466=85, srv597945969=84, srv1626766036=32, srv1038324665=1, srv490025992=74, srv558433630=81, srv1324117890=13, srv812495635=93, srv567949125=83, srv154419449=25, srv436293677=68, srv518623878=77, srv55710863=79, srv1313234554=12, srv221927050=58, srv745874507=89, srv2035840255=50, srv894219606=95, srv1209459637=5, srv1961370827=49, srv1082755414=2, srv1376520336=16, srv185110058=41, srv752294556=90, srv1597357863=28, srv1812288523=40, srv676821733=86, srv558232378=80, srv2090166666=52, srv473348208=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:53:20,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:53:20,539 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:53:20,540 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:53:20,541 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:53:20,542 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-19 04:53:20,543 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-19 04:53:20,544 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-19 04:53:20,545 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-19 04:53:20,546 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-19 04:53:20,547 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-19 04:53:20,548 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-19 04:53:20,549 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-19 04:53:20,550 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-19 04:53:20,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-19 04:53:20,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-19 04:53:20,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-19 04:53:20,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-19 04:53:20,551 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-19 04:53:20,708 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019478805618229192); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027389201342287256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019478805618229192); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04533333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:20,708 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-19 04:53:20,708 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016452092647534857, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019478805618229192); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.027389201342287256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019478805618229192); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0135); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04533333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-19 04:53:21,062 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 488 ms to try 5000 different iterations.  Found a solution that moves 1041 regions; Going from a computed imbalance of 0.016452092647534857 to a new imbalance of 8.70771844406548E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054341389903924614); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006011586269238994); MoveCostFunction : (multiplier=7.0, imbalance=0.11566666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054341389903924614); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:21,065 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1376520336:82 , srv413024907:82 , srv812495635:82 , srv889862470:82 , srv894219606:82 , srv1156690619:83 , srv1313234554:83 , srv1812288523:83 , srv350692199:83 , srv366880201:83 , srv445983686:83 , srv55710863:83 , srv1244013594:84 , srv154419449:84 , srv1626766036:84 , srv1809601176:84 , srv1897129028:84 , srv1915502160:84 , srv289964572:84 , srv317807086:84 , srv553630384:84 , srv558433630:84 , srv566578945:84 , srv798974490:84 , srv960762199:84 , srv1027154246:85 , srv1038324665:85 , srv1217398726:85 , srv121846343:85 , srv1220159557:85 , srv1256096544:85 , srv1324117890:85 , srv1371666944:85 , srv1435728896:85 , srv1456972594:85 , srv1492029926:85 , srv1607555193:85 , srv1651888318:85 , srv1883987140:85 , srv2035840255:85 , srv2082513551:85 , srv2090166666:85 , srv221927050:85 , srv2467381:85 , srv436293677:85 , srv473348208:85 , srv473375391:85 , srv518623878:85 , srv558232378:85 , srv597945969:85 , srv676821733:85 , srv767734812:85 , srv1082755414:86 , srv1282157263:86 , srv1430803481:86 , srv1441778036:86 , srv1467099224:86 , srv1568243477:86 , srv158724490:86 , srv1597357863:86 , srv1607515749:86 , srv1751937367:86 , srv185110058:86 , srv1906546949:86 , srv193235380:86 , srv2095201424:86 , srv2098504385:86 , srv211442704:86 , srv220170697:86 , srv284676685:86 , srv350014950:86 , srv460684685:86 , srv466590853:86 , srv490025992:86 , srv567949125:86 , srv61790466:86 , srv73159367:86 , srv752294556:86 , srv953207268:86 , srv97610462:86 , srv984416122:86 , srv1192040911:87 , srv1209459637:87 , srv1328640653:87 , srv147975630:87 , srv1616915136:87 , srv1674573702:87 , srv1728317916:87 , srv190042652:87 , srv1920635465:87 , srv1961370827:87 , srv2123302318:87 , srv273395332:87 , srv50803142:87 , srv515342143:87 , srv745874507:87 , srv1378375528:88 , srv1790566797:88 , srv682425885:88 , srv1700042534:559 }
2023-09-19 04:53:21,071 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:53:21,073 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1790566797=38, srv466590853=71, srv147975630=23, srv2082513551=51, srv960762199=97, srv350692199=65, srv413024907=67, srv953207268=96, srv1651888318=33, srv97610462=98, srv1674573702=34, srv190042652=44, srv2098504385=54, srv1920635465=47, srv1192040911=4, srv2095201424=53, srv220170697=57, srv1456972594=21, srv984416122=99, srv445983686=69, srv1700042534=35, srv1435728896=19, srv1027154246=0, srv158724490=27, srv1492029926=24, srv1156690619=3, srv1378375528=17, srv121846343=7, srv2467381=59, srv284676685=61, srv1616915136=31, srv1897129028=43, srv1217398726=6, srv350014950=64, srv1728317916=36, srv515342143=76, srv767734812=91, srv1371666944=15, srv1915502160=46, srv682425885=87, srv1906546949=45, srv1244013594=9, srv473375391=73, srv193235380=48, srv1568243477=26, srv1256096544=10, srv1282157263=11, srv889862470=94, srv1220159557=8, srv1441778036=20, srv211442704=55, srv1883987140=42, srv1607555193=30, srv460684685=70, srv1328640653=14, srv273395332=60, srv553630384=78, srv1467099224=22, srv566578945=82, srv73159367=88, srv2123302318=56, srv1430803481=18, srv289964572=62, srv798974490=92, srv1809601176=39, srv1607515749=29, srv317807086=63, srv1751937367=37, srv366880201=66, srv50803142=75, srv61790466=85, srv597945969=84, srv1626766036=32, srv1038324665=1, srv490025992=74, srv558433630=81, srv1324117890=13, srv812495635=93, srv567949125=83, srv154419449=25, srv436293677=68, srv518623878=77, srv55710863=79, srv1313234554=12, srv221927050=58, srv745874507=89, srv2035840255=50, srv894219606=95, srv1209459637=5, srv1961370827=49, srv1082755414=2, srv1376520336=16, srv185110058=41, srv752294556=90, srv1597357863=28, srv1812288523=40, srv676821733=86, srv558232378=80, srv2090166666=52, srv473348208=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:53:21,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:53:21,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:53:21,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:53:21,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:53:21,092 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:53:21,093 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:53:21,094 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-19 04:53:21,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-19 04:53:21,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-19 04:53:21,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-19 04:53:21,098 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-19 04:53:21,099 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-19 04:53:21,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-19 04:53:21,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-19 04:53:21,102 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-19 04:53:21,253 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054341389903924614); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006011586269238994); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054341389903924614); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:21,253 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-19 04:53:21,256 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.634826066105446E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054341389903924614); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006011586269238994); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054341389903924614); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-19 04:53:21,643 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 527 ms to try 5000 different iterations.  Found a solution that moves 274 regions; Going from a computed imbalance of 8.634826066105446E-4 to a new imbalance of 1.7970319303349155E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.030444444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:21,644 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1209459637:83 , srv190042652:83 , srv366880201:83 , srv682425885:83 , srv1082755414:84 , srv121846343:84 , srv1244013594:84 , srv1313234554:84 , srv1324117890:84 , srv1328640653:84 , srv1376520336:84 , srv1378375528:84 , srv1430803481:84 , srv1441778036:84 , srv1456972594:84 , srv1467099224:84 , srv147975630:84 , srv1568243477:84 , srv158724490:84 , srv1607555193:84 , srv1626766036:84 , srv1674573702:84 , srv1728317916:84 , srv1751937367:84 , srv1790566797:84 , srv1897129028:84 , srv1906546949:84 , srv1920635465:84 , srv193235380:84 , srv2035840255:84 , srv2082513551:84 , srv2095201424:84 , srv2098504385:84 , srv211442704:84 , srv2123302318:84 , srv2467381:84 , srv273395332:84 , srv289964572:84 , srv350692199:84 , srv445983686:84 , srv466590853:84 , srv473348208:84 , srv473375391:84 , srv50803142:84 , srv558232378:84 , srv566578945:84 , srv752294556:84 , srv767734812:84 , srv812495635:84 , srv953207268:84 , srv97610462:84 , srv984416122:84 , srv1027154246:85 , srv1038324665:85 , srv1156690619:85 , srv1192040911:85 , srv1217398726:85 , srv1220159557:85 , srv1256096544:85 , srv1282157263:85 , srv1371666944:85 , srv1435728896:85 , srv1492029926:85 , srv154419449:85 , srv1597357863:85 , srv1607515749:85 , srv1616915136:85 , srv1651888318:85 , srv1809601176:85 , srv1812288523:85 , srv185110058:85 , srv1883987140:85 , srv1961370827:85 , srv220170697:85 , srv221927050:85 , srv284676685:85 , srv317807086:85 , srv350014950:85 , srv413024907:85 , srv436293677:85 , srv460684685:85 , srv490025992:85 , srv515342143:85 , srv518623878:85 , srv55710863:85 , srv558433630:85 , srv567949125:85 , srv597945969:85 , srv61790466:85 , srv676821733:85 , srv73159367:85 , srv745874507:85 , srv798974490:85 , srv894219606:85 , srv1915502160:86 , srv2090166666:86 , srv553630384:86 , srv889862470:86 , srv960762199:86 , srv1700042534:636 }
2023-09-19 04:53:21,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:53:21,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1790566797=38, srv466590853=71, srv147975630=23, srv2082513551=51, srv960762199=97, srv350692199=65, srv413024907=67, srv953207268=96, srv1651888318=33, srv97610462=98, srv1674573702=34, srv190042652=44, srv2098504385=54, srv1920635465=47, srv1192040911=4, srv2095201424=53, srv220170697=57, srv1456972594=21, srv984416122=99, srv445983686=69, srv1700042534=35, srv1435728896=19, srv1027154246=0, srv158724490=27, srv1492029926=24, srv1156690619=3, srv1378375528=17, srv121846343=7, srv2467381=59, srv284676685=61, srv1616915136=31, srv1897129028=43, srv1217398726=6, srv350014950=64, srv1728317916=36, srv515342143=76, srv767734812=91, srv1371666944=15, srv1915502160=46, srv682425885=87, srv1906546949=45, srv1244013594=9, srv473375391=73, srv193235380=48, srv1568243477=26, srv1256096544=10, srv1282157263=11, srv889862470=94, srv1220159557=8, srv1441778036=20, srv211442704=55, srv1883987140=42, srv1607555193=30, srv460684685=70, srv1328640653=14, srv273395332=60, srv553630384=78, srv1467099224=22, srv566578945=82, srv73159367=88, srv2123302318=56, srv1430803481=18, srv289964572=62, srv798974490=92, srv1809601176=39, srv1607515749=29, srv317807086=63, srv1751937367=37, srv366880201=66, srv50803142=75, srv61790466=85, srv597945969=84, srv1626766036=32, srv1038324665=1, srv490025992=74, srv558433630=81, srv1324117890=13, srv812495635=93, srv567949125=83, srv154419449=25, srv436293677=68, srv518623878=77, srv55710863=79, srv1313234554=12, srv221927050=58, srv745874507=89, srv2035840255=50, srv894219606=95, srv1209459637=5, srv1961370827=49, srv1082755414=2, srv1376520336=16, srv185110058=41, srv752294556=90, srv1597357863=28, srv1812288523=40, srv676821733=86, srv558232378=80, srv2090166666=52, srv473348208=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:53:21,659 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-19 04:53:21,660 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-19 04:53:21,661 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-19 04:53:21,662 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-19 04:53:21,663 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-19 04:53:21,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-19 04:53:21,768 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.605173029652406E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:21,768 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1209459637:83 , srv190042652:83 , srv366880201:83 , srv682425885:83 , srv1082755414:84 , srv121846343:84 , srv1244013594:84 , srv1313234554:84 , srv1324117890:84 , srv1328640653:84 , srv1376520336:84 , srv1378375528:84 , srv1430803481:84 , srv1441778036:84 , srv1456972594:84 , srv1467099224:84 , srv147975630:84 , srv1568243477:84 , srv158724490:84 , srv1607555193:84 , srv1626766036:84 , srv1674573702:84 , srv1728317916:84 , srv1751937367:84 , srv1790566797:84 , srv1897129028:84 , srv1906546949:84 , srv1920635465:84 , srv193235380:84 , srv2035840255:84 , srv2082513551:84 , srv2095201424:84 , srv2098504385:84 , srv211442704:84 , srv2123302318:84 , srv2467381:84 , srv273395332:84 , srv289964572:84 , srv350692199:84 , srv445983686:84 , srv466590853:84 , srv473348208:84 , srv473375391:84 , srv50803142:84 , srv558232378:84 , srv566578945:84 , srv752294556:84 , srv767734812:84 , srv812495635:84 , srv953207268:84 , srv97610462:84 , srv984416122:84 , srv1027154246:85 , srv1038324665:85 , srv1156690619:85 , srv1192040911:85 , srv1217398726:85 , srv1220159557:85 , srv1256096544:85 , srv1282157263:85 , srv1371666944:85 , srv1435728896:85 , srv1492029926:85 , srv154419449:85 , srv1597357863:85 , srv1607515749:85 , srv1616915136:85 , srv1651888318:85 , srv1809601176:85 , srv1812288523:85 , srv185110058:85 , srv1883987140:85 , srv1961370827:85 , srv220170697:85 , srv221927050:85 , srv284676685:85 , srv317807086:85 , srv350014950:85 , srv413024907:85 , srv436293677:85 , srv460684685:85 , srv490025992:85 , srv515342143:85 , srv518623878:85 , srv55710863:85 , srv558433630:85 , srv567949125:85 , srv597945969:85 , srv61790466:85 , srv676821733:85 , srv73159367:85 , srv745874507:85 , srv798974490:85 , srv894219606:85 , srv1915502160:86 , srv2090166666:86 , srv553630384:86 , srv889862470:86 , srv960762199:86 , srv1700042534:636 }
2023-09-19 04:53:21,798 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=367 (was 367), ProcessCount=170 (was 170), AvailableMemoryMB=10148 (was 10791)
2023-09-19 04:53:21,807 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=367, ProcessCount=170, AvailableMemoryMB=10148
2023-09-19 04:53:21,808 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-19 04:53:21,808 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-19 04:53:21,816 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv102053062:84 , srv1108316093:84 , srv11194621:84 , srv1152705021:84 , srv1304059026:84 , srv1321738482:84 , srv1505941865:84 , srv1570952348:84 , srv1621646657:84 , srv1692098637:84 , srv1703675083:84 , srv1706281105:84 , srv1746813759:84 , srv1955538357:84 , srv2064540751:84 , srv2131737764:84 , srv288689250:84 , srv389995197:84 , srv492237920:84 , srv502825766:84 , srv558328386:84 , srv589878063:84 , srv629878209:84 , srv684287426:84 , srv750403664:84 , srv916972121:84 , srv922043201:84 , srv963062497:84 , srv980195015:84 , srv752593693:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-19 04:53:21,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:53:21,817 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1304059026=4, srv1692098637=9, srv288689250=16, srv2064540751=14, srv980195015=29, srv1321738482=5, srv102053062=0, srv589878063=21, srv2131737764=15, srv922043201=27, srv1152705021=3, srv389995197=17, srv1505941865=6, srv963062497=28, srv558328386=20, srv1108316093=1, srv1706281105=11, srv752593693=25, srv1746813759=12, srv916972121=26, srv750403664=24, srv492237920=18, srv1955538357=13, srv629878209=22, srv1703675083=10, srv684287426=23, srv1570952348=7, srv1621646657=8, srv502825766=19, srv11194621=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-19 04:53:21,842 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0536862316712567, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:53:21,842 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912362206827261, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0536862316712567, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-19 04:55:21,829 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5942954 different iterations.  Found a solution that moves 1853 regions; Going from a computed imbalance of 0.9912362206827261 to a new imbalance of 4.324994439959734E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6862962962962963, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:55:21,835 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv102053062:90 , srv1108316093:90 , srv11194621:90 , srv1152705021:90 , srv1304059026:90 , srv1321738482:90 , srv1505941865:90 , srv1570952348:90 , srv1621646657:90 , srv1692098637:90 , srv1703675083:90 , srv1706281105:90 , srv1746813759:90 , srv1955538357:90 , srv2064540751:90 , srv2131737764:90 , srv288689250:90 , srv389995197:90 , srv492237920:90 , srv502825766:90 , srv558328386:90 , srv589878063:90 , srv629878209:90 , srv684287426:90 , srv750403664:90 , srv752593693:90 , srv916972121:90 , srv922043201:90 , srv963062497:90 , srv980195015:90 }
2023-09-19 04:55:21,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-19 04:55:21,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1304059026=4, srv1692098637=9, srv288689250=16, srv2064540751=14, srv980195015=29, srv1321738482=5, srv102053062=0, srv589878063=21, srv2131737764=15, srv922043201=27, srv1152705021=3, srv389995197=17, srv1505941865=6, srv963062497=28, srv558328386=20, srv1108316093=1, srv1706281105=11, srv752593693=25, srv1746813759=12, srv916972121=26, srv750403664=24, srv492237920=18, srv1955538357=13, srv629878209=22, srv1703675083=10, srv684287426=23, srv1570952348=7, srv1621646657=8, srv502825766=19, srv11194621=2} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-19 04:55:21,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-19 04:55:21,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-19 04:55:21,839 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-19 04:55:21,840 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-19 04:55:21,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-19 04:55:21,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-19 04:55:21,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-19 04:55:21,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-19 04:55:21,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-19 04:55:21,841 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-19 04:55:21,852 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-19 04:55:21,852 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv102053062:90 , srv1108316093:90 , srv11194621:90 , srv1152705021:90 , srv1304059026:90 , srv1321738482:90 , srv1505941865:90 , srv1570952348:90 , srv1621646657:90 , srv1692098637:90 , srv1703675083:90 , srv1706281105:90 , srv1746813759:90 , srv1955538357:90 , srv2064540751:90 , srv2131737764:90 , srv288689250:90 , srv389995197:90 , srv492237920:90 , srv502825766:90 , srv558328386:90 , srv589878063:90 , srv629878209:90 , srv684287426:90 , srv750403664:90 , srv752593693:90 , srv916972121:90 , srv922043201:90 , srv963062497:90 , srv980195015:90 }
2023-09-19 04:55:21,865 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=156 (was 367), ProcessCount=169 (was 170), AvailableMemoryMB=9355 (was 10148)