2023-09-24 08:53:14,819 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-24 08:53:15,149 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-24 08:53:15,267 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-24 08:53:15,333 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-24 08:53:15,340 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-24 08:53:15,354 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=171, AvailableMemoryMB=10109
2023-09-24 08:53:15,357 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-24 08:53:15,357 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-24 08:53:15,519 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035735470:84 , srv10580543:84 , srv1058193955:84 , srv1059282567:84 , srv1080589145:84 , srv108918427:84 , srv1097961769:84 , srv1098354050:84 , srv111323899:84 , srv1122874351:84 , srv112788601:84 , srv1138137831:84 , srv1151742588:84 , srv120320351:84 , srv1262112443:84 , srv1286363750:84 , srv1296747128:84 , srv1330223342:84 , srv1349299138:84 , srv135743265:84 , srv1382999493:84 , srv1438766086:84 , srv1441669762:84 , srv1446741366:84 , srv1456251033:84 , srv1460897749:84 , srv1466886555:84 , srv1479529106:84 , srv1480600183:84 , srv1491533591:84 , srv1555929688:84 , srv1637290291:84 , srv1669590542:84 , srv168920599:84 , srv1700223846:84 , srv1705109999:84 , srv1722977996:84 , srv1732007244:84 , srv1756485384:84 , srv1757664417:84 , srv1769229551:84 , srv1783952282:84 , srv1818746009:84 , srv1835432502:84 , srv186790685:84 , srv1873038188:84 , srv1900772460:84 , srv1926031882:84 , srv1955888818:84 , srv1958978805:84 , srv1961351483:84 , srv1970393871:84 , srv197196532:84 , srv2036480623:84 , srv205426767:84 , srv2056233397:84 , srv2104519869:84 , srv2119777377:84 , srv2141084081:84 , srv2142470779:84 , srv2145528765:84 , srv234333668:84 , srv267315532:84 , srv268245727:84 , srv337457506:84 , srv372527811:84 , srv373472940:84 , srv382166385:84 , srv384439738:84 , srv384972826:84 , srv40640022:84 , srv407683054:84 , srv447715617:84 , srv485299373:84 , srv513969371:84 , srv523907576:84 , srv525162889:84 , srv528290806:84 , srv572867635:84 , srv579918724:84 , srv596923983:84 , srv601086492:84 , srv618976154:84 , srv624641128:84 , srv670531863:84 , srv67354313:84 , srv673726989:84 , srv701583895:84 , srv710979266:84 , srv729768264:84 , srv730211645:84 , srv731154348:84 , srv762673198:84 , srv805264420:84 , srv81087729:84 , srv819138755:84 , srv819546707:84 , srv865314586:84 , srv949981234:84 , srv1078154944:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-24 08:53:15,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:15,535 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1286363750=16, srv2036480623=54, srv407683054=72, srv2141084081=59, srv268245727=64, srv120320351=14, srv1873038188=46, srv819138755=96, srv2119777377=58, srv1059282567=3, srv513969371=75, srv670531863=85, srv197196532=53, srv528290806=78, srv2104519869=57, srv81087729=95, srv1035735470=0, srv1722977996=37, srv108918427=6, srv579918724=80, srv168920599=34, srv865314586=98, srv1330223342=18, srv372527811=66, srv1138137831=12, srv1818746009=43, srv1835432502=44, srv1080589145=5, srv1122874351=10, srv135743265=20, srv1098354050=8, srv2056233397=56, srv1456251033=25, srv1446741366=24, srv710979266=89, srv1480600183=29, srv1441669762=23, srv525162889=77, srv1555929688=31, srv205426767=55, srv485299373=74, srv618976154=83, srv1058193955=2, srv1926031882=48, srv186790685=45, srv1732007244=38, srv1669590542=33, srv40640022=71, srv596923983=81, srv1955888818=49, srv1757664417=40, srv572867635=79, srv1466886555=27, srv1637290291=32, srv731154348=92, srv1382999493=21, srv1970393871=52, srv1479529106=28, srv1900772460=47, srv1460897749=26, srv730211645=91, srv673726989=87, srv382166385=68, srv1349299138=19, srv2142470779=60, srv1078154944=4, srv624641128=84, srv373472940=67, srv67354313=86, srv1097961769=7, srv384972826=70, srv819546707=97, srv1769229551=41, srv337457506=65, srv2145528765=61, srv805264420=94, srv384439738=69, srv1262112443=15, srv1151742588=13, srv267315532=63, srv1961351483=51, srv701583895=88, srv10580543=1, srv1296747128=17, srv729768264=90, srv1756485384=39, srv1705109999=36, srv1783952282=42, srv949981234=99, srv112788601=11, srv1491533591=30, srv1438766086=22, srv1958978805=50, srv234333668=62, srv601086492=82, srv762673198=93, srv1700223846=35, srv523907576=76, srv111323899=9, srv447715617=73} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:15,607 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:15,608 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:15,609 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 08:53:15,610 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 08:53:15,611 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 08:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 08:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 08:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 08:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 08:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 08:53:15,612 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 08:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 08:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 08:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 08:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 08:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 08:53:15,613 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 08:53:15,614 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 08:53:15,615 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 08:53:15,616 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 08:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 08:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 08:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 08:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 08:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 08:53:15,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 08:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 08:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 08:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 08:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 08:53:15,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 08:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 08:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 08:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 08:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 08:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 08:53:15,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 08:53:15,974 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:15,974 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 08:53:15,975 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 08:53:16,616 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 969 ms to try 5000 different iterations.  Found a solution that moves 4728 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1783710145963993. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04016520318107304, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06050294257807681, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5253333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04016520318107304, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1735, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:16,633 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1059282567:82 , srv762673198:82 , srv108918427:83 , srv111323899:83 , srv1480600183:83 , srv1955888818:83 , srv2056233397:83 , srv2119777377:83 , srv384972826:83 , srv528290806:83 , srv670531863:83 , srv729768264:83 , srv1441669762:84 , srv1456251033:84 , srv1637290291:84 , srv1873038188:84 , srv1961351483:84 , srv1970393871:84 , srv268245727:84 , srv372527811:84 , srv373472940:84 , srv81087729:84 , srv1035735470:85 , srv1080589145:85 , srv1097961769:85 , srv1138137831:85 , srv1151742588:85 , srv1349299138:85 , srv1382999493:85 , srv1438766086:85 , srv1700223846:85 , srv1732007244:85 , srv1818746009:85 , srv2142470779:85 , srv2145528765:85 , srv234333668:85 , srv337457506:85 , srv572867635:85 , srv618976154:85 , srv805264420:85 , srv1058193955:86 , srv1286363750:86 , srv135743265:86 , srv1491533591:86 , srv1669590542:86 , srv1835432502:86 , srv186790685:86 , srv1958978805:86 , srv2036480623:86 , srv2104519869:86 , srv40640022:86 , srv601086492:86 , srv624641128:86 , srv67354313:86 , srv673726989:86 , srv701583895:86 , srv730211645:86 , srv731154348:86 , srv1122874351:87 , srv1460897749:87 , srv168920599:87 , srv1705109999:87 , srv2141084081:87 , srv384439738:87 , srv407683054:87 , srv513969371:87 , srv525162889:87 , srv579918724:87 , srv596923983:87 , srv1098354050:88 , srv120320351:88 , srv1296747128:88 , srv1330223342:88 , srv1466886555:88 , srv1555929688:88 , srv1783952282:88 , srv1900772460:88 , srv205426767:88 , srv523907576:88 , srv819546707:88 , srv865314586:88 , srv949981234:88 , srv112788601:89 , srv1446741366:89 , srv1722977996:89 , srv1757664417:89 , srv1769229551:89 , srv1926031882:89 , srv710979266:89 , srv1262112443:90 , srv1479529106:90 , srv10580543:91 , srv1756485384:91 , srv447715617:91 , srv267315532:92 , srv197196532:93 , srv382166385:93 , srv485299373:93 , srv819138755:93 , srv1078154944:447 }
2023-09-24 08:53:16,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:16,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1286363750=16, srv2036480623=54, srv407683054=72, srv2141084081=59, srv268245727=64, srv120320351=14, srv1873038188=46, srv819138755=96, srv2119777377=58, srv1059282567=3, srv513969371=75, srv670531863=85, srv197196532=53, srv528290806=78, srv2104519869=57, srv81087729=95, srv1035735470=0, srv1722977996=37, srv108918427=6, srv579918724=80, srv168920599=34, srv865314586=98, srv1330223342=18, srv372527811=66, srv1138137831=12, srv1818746009=43, srv1835432502=44, srv1080589145=5, srv1122874351=10, srv135743265=20, srv1098354050=8, srv2056233397=56, srv1456251033=25, srv1446741366=24, srv710979266=89, srv1480600183=29, srv1441669762=23, srv525162889=77, srv1555929688=31, srv205426767=55, srv485299373=74, srv618976154=83, srv1058193955=2, srv1926031882=48, srv186790685=45, srv1732007244=38, srv1669590542=33, srv40640022=71, srv596923983=81, srv1955888818=49, srv1757664417=40, srv572867635=79, srv1466886555=27, srv1637290291=32, srv731154348=92, srv1382999493=21, srv1970393871=52, srv1479529106=28, srv1900772460=47, srv1460897749=26, srv730211645=91, srv673726989=87, srv382166385=68, srv1349299138=19, srv2142470779=60, srv1078154944=4, srv624641128=84, srv373472940=67, srv67354313=86, srv1097961769=7, srv384972826=70, srv819546707=97, srv1769229551=41, srv337457506=65, srv2145528765=61, srv805264420=94, srv384439738=69, srv1262112443=15, srv1151742588=13, srv267315532=63, srv1961351483=51, srv701583895=88, srv10580543=1, srv1296747128=17, srv729768264=90, srv1756485384=39, srv1705109999=36, srv1783952282=42, srv949981234=99, srv112788601=11, srv1491533591=30, srv1438766086=22, srv1958978805=50, srv234333668=62, srv601086492=82, srv762673198=93, srv1700223846=35, srv523907576=76, srv111323899=9, srv447715617=73} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:16,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:16,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:16,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:16,670 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:16,671 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 08:53:16,672 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 08:53:16,673 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 08:53:16,674 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 08:53:16,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 08:53:16,676 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 08:53:16,677 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 08:53:16,678 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 08:53:16,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 08:53:16,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 08:53:16,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 08:53:16,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 08:53:16,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 08:53:16,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 08:53:16,892 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04016520318107304, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06050294257807681, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04016520318107304, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1735, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:16,893 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 08:53:16,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17833790843280709, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04016520318107304, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06050294257807681, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04016520318107304, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1735, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24075, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 08:53:17,234 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 528 ms to try 5000 different iterations.  Found a solution that moves 2621 regions; Going from a computed imbalance of 0.17833790843280709 to a new imbalance of 0.013612254841866489. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020701020166640178); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0377896594505917, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2912222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020701020166640178); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0105); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:17,251 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv10580543:79 , srv197196532:79 , srv1756485384:80 , srv1769229551:81 , srv267315532:81 , srv382166385:81 , srv447715617:81 , srv485299373:81 , srv819138755:81 , srv1262112443:82 , srv865314586:82 , srv1098354050:83 , srv1296747128:83 , srv1479529106:83 , srv1757664417:83 , srv710979266:83 , srv949981234:83 , srv1097961769:84 , srv112788601:84 , srv1460897749:84 , srv1466886555:84 , srv1555929688:84 , srv1722977996:84 , srv1783952282:84 , srv1835432502:84 , srv205426767:84 , srv523907576:84 , srv731154348:84 , srv120320351:85 , srv1446741366:85 , srv1491533591:85 , srv1669590542:85 , srv1705109999:85 , srv1900772460:85 , srv1926031882:85 , srv2036480623:85 , srv2141084081:85 , srv2142470779:85 , srv384439738:85 , srv40640022:85 , srv596923983:85 , srv601086492:85 , srv624641128:85 , srv701583895:85 , srv819546707:85 , srv1035735470:86 , srv1058193955:86 , srv1138137831:86 , srv1286363750:86 , srv1349299138:86 , srv1382999493:86 , srv168920599:86 , srv1700223846:86 , srv1732007244:86 , srv1818746009:86 , srv513969371:86 , srv618976154:86 , srv1122874351:87 , srv1151742588:87 , srv1330223342:87 , srv1438766086:87 , srv1456251033:87 , srv1958978805:87 , srv2145528765:87 , srv234333668:87 , srv268245727:87 , srv373472940:87 , srv407683054:87 , srv572867635:87 , srv579918724:87 , srv673726989:87 , srv730211645:87 , srv805264420:87 , srv81087729:87 , srv1080589145:88 , srv108918427:88 , srv1441669762:88 , srv1637290291:88 , srv186790685:88 , srv1873038188:88 , srv1970393871:88 , srv2104519869:88 , srv337457506:88 , srv384972826:88 , srv67354313:88 , srv1059282567:89 , srv111323899:89 , srv135743265:89 , srv1955888818:89 , srv2056233397:89 , srv372527811:89 , srv525162889:89 , srv729768264:89 , srv1480600183:90 , srv1961351483:90 , srv528290806:90 , srv670531863:90 , srv2119777377:91 , srv762673198:92 , srv1078154944:511 }
2023-09-24 08:53:17,258 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:17,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1286363750=16, srv2036480623=54, srv407683054=72, srv2141084081=59, srv268245727=64, srv120320351=14, srv1873038188=46, srv819138755=96, srv2119777377=58, srv1059282567=3, srv513969371=75, srv670531863=85, srv197196532=53, srv528290806=78, srv2104519869=57, srv81087729=95, srv1035735470=0, srv1722977996=37, srv108918427=6, srv579918724=80, srv168920599=34, srv865314586=98, srv1330223342=18, srv372527811=66, srv1138137831=12, srv1818746009=43, srv1835432502=44, srv1080589145=5, srv1122874351=10, srv135743265=20, srv1098354050=8, srv2056233397=56, srv1456251033=25, srv1446741366=24, srv710979266=89, srv1480600183=29, srv1441669762=23, srv525162889=77, srv1555929688=31, srv205426767=55, srv485299373=74, srv618976154=83, srv1058193955=2, srv1926031882=48, srv186790685=45, srv1732007244=38, srv1669590542=33, srv40640022=71, srv596923983=81, srv1955888818=49, srv1757664417=40, srv572867635=79, srv1466886555=27, srv1637290291=32, srv731154348=92, srv1382999493=21, srv1970393871=52, srv1479529106=28, srv1900772460=47, srv1460897749=26, srv730211645=91, srv673726989=87, srv382166385=68, srv1349299138=19, srv2142470779=60, srv1078154944=4, srv624641128=84, srv373472940=67, srv67354313=86, srv1097961769=7, srv384972826=70, srv819546707=97, srv1769229551=41, srv337457506=65, srv2145528765=61, srv805264420=94, srv384439738=69, srv1262112443=15, srv1151742588=13, srv267315532=63, srv1961351483=51, srv701583895=88, srv10580543=1, srv1296747128=17, srv729768264=90, srv1756485384=39, srv1705109999=36, srv1783952282=42, srv949981234=99, srv112788601=11, srv1491533591=30, srv1438766086=22, srv1958978805=50, srv234333668=62, srv601086492=82, srv762673198=93, srv1700223846=35, srv523907576=76, srv111323899=9, srv447715617=73} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:17,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:17,288 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:17,289 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:17,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 08:53:17,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 08:53:17,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 08:53:17,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 08:53:17,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 08:53:17,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 08:53:17,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 08:53:17,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 08:53:17,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 08:53:17,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 08:53:17,469 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020701020166640178); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0377896594505917, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020701020166640178); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0105); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:17,488 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 08:53:17,489 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013593902207607771, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020701020166640178); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0377896594505917, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020701020166640178); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0105); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 08:53:17,791 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 468 ms to try 5000 different iterations.  Found a solution that moves 975 regions; Going from a computed imbalance of 0.013593902207607771 to a new imbalance of 4.6017818506977184E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054295474061417486); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009904142134751739); MoveCostFunction : (multiplier=7.0, imbalance=0.10833333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054295474061417486); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:17,794 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv525162889:81 , srv1330223342:82 , srv135743265:82 , srv1122874351:83 , srv1446741366:83 , srv186790685:83 , srv2119777377:83 , srv579918724:83 , srv67354313:83 , srv1080589145:84 , srv120320351:84 , srv168920599:84 , srv1900772460:84 , srv1926031882:84 , srv1961351483:84 , srv2104519869:84 , srv372527811:84 , srv382166385:84 , srv407683054:84 , srv485299373:84 , srv513969371:84 , srv670531863:84 , srv730211645:84 , srv762673198:84 , srv819138755:84 , srv111323899:85 , srv112788601:85 , srv1262112443:85 , srv1441669762:85 , srv1466886555:85 , srv1480600183:85 , srv1705109999:85 , srv1722977996:85 , srv1783952282:85 , srv205426767:85 , srv2056233397:85 , srv2141084081:85 , srv2145528765:85 , srv234333668:85 , srv267315532:85 , srv337457506:85 , srv523907576:85 , srv528290806:85 , srv572867635:85 , srv596923983:85 , srv673726989:85 , srv729768264:85 , srv819546707:85 , srv1035735470:86 , srv1059282567:86 , srv1098354050:86 , srv1138137831:86 , srv1151742588:86 , srv1296747128:86 , srv1349299138:86 , srv1382999493:86 , srv1438766086:86 , srv1479529106:86 , srv1491533591:86 , srv1555929688:86 , srv1637290291:86 , srv1700223846:86 , srv1732007244:86 , srv1756485384:86 , srv1757664417:86 , srv1818746009:86 , srv1873038188:86 , srv1955888818:86 , srv1958978805:86 , srv1970393871:86 , srv197196532:86 , srv268245727:86 , srv384439738:86 , srv447715617:86 , srv601086492:86 , srv618976154:86 , srv701583895:86 , srv805264420:86 , srv10580543:87 , srv1058193955:87 , srv108918427:87 , srv1286363750:87 , srv1456251033:87 , srv1669590542:87 , srv1769229551:87 , srv1835432502:87 , srv2036480623:87 , srv2142470779:87 , srv373472940:87 , srv384972826:87 , srv40640022:87 , srv624641128:87 , srv710979266:87 , srv731154348:87 , srv81087729:87 , srv865314586:87 , srv949981234:87 , srv1460897749:88 , srv1097961769:89 , srv1078154944:548 }
2023-09-24 08:53:17,800 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:17,801 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1286363750=16, srv2036480623=54, srv407683054=72, srv2141084081=59, srv268245727=64, srv120320351=14, srv1873038188=46, srv819138755=96, srv2119777377=58, srv1059282567=3, srv513969371=75, srv670531863=85, srv197196532=53, srv528290806=78, srv2104519869=57, srv81087729=95, srv1035735470=0, srv1722977996=37, srv108918427=6, srv579918724=80, srv168920599=34, srv865314586=98, srv1330223342=18, srv372527811=66, srv1138137831=12, srv1818746009=43, srv1835432502=44, srv1080589145=5, srv1122874351=10, srv135743265=20, srv1098354050=8, srv2056233397=56, srv1456251033=25, srv1446741366=24, srv710979266=89, srv1480600183=29, srv1441669762=23, srv525162889=77, srv1555929688=31, srv205426767=55, srv485299373=74, srv618976154=83, srv1058193955=2, srv1926031882=48, srv186790685=45, srv1732007244=38, srv1669590542=33, srv40640022=71, srv596923983=81, srv1955888818=49, srv1757664417=40, srv572867635=79, srv1466886555=27, srv1637290291=32, srv731154348=92, srv1382999493=21, srv1970393871=52, srv1479529106=28, srv1900772460=47, srv1460897749=26, srv730211645=91, srv673726989=87, srv382166385=68, srv1349299138=19, srv2142470779=60, srv1078154944=4, srv624641128=84, srv373472940=67, srv67354313=86, srv1097961769=7, srv384972826=70, srv819546707=97, srv1769229551=41, srv337457506=65, srv2145528765=61, srv805264420=94, srv384439738=69, srv1262112443=15, srv1151742588=13, srv267315532=63, srv1961351483=51, srv701583895=88, srv10580543=1, srv1296747128=17, srv729768264=90, srv1756485384=39, srv1705109999=36, srv1783952282=42, srv949981234=99, srv112788601=11, srv1491533591=30, srv1438766086=22, srv1958978805=50, srv234333668=62, srv601086492=82, srv762673198=93, srv1700223846=35, srv523907576=76, srv111323899=9, srv447715617=73} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:17,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:17,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 08:53:17,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 08:53:17,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 08:53:17,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 08:53:17,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 08:53:17,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 08:53:17,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 08:53:17,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 08:53:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 08:53:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 08:53:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 08:53:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 08:53:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 08:53:17,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 08:53:17,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054295474061417486); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009904142134751739); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054295474061417486); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:17,947 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 08:53:17,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.5335108915132484E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0054295474061417486); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009904142134751739); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0054295474061417486); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003416666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 08:53:18,301 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 459 ms to try 5000 different iterations.  Found a solution that moves 237 regions; Going from a computed imbalance of 4.5335108915132484E-4 to a new imbalance of 2.662474365223691E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.026333333333333334, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:18,302 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1058193955:83 , srv1286363750:83 , srv1460897749:83 , srv1479529106:83 , srv1926031882:83 , srv1958978805:83 , srv1970393871:83 , srv2036480623:83 , srv81087729:83 , srv1035735470:84 , srv108918427:84 , srv1097961769:84 , srv112788601:84 , srv1151742588:84 , srv1480600183:84 , srv1555929688:84 , srv1637290291:84 , srv1669590542:84 , srv1722977996:84 , srv1757664417:84 , srv1873038188:84 , srv1955888818:84 , srv1961351483:84 , srv2104519869:84 , srv267315532:84 , srv337457506:84 , srv373472940:84 , srv382166385:84 , srv384439738:84 , srv384972826:84 , srv40640022:84 , srv407683054:84 , srv447715617:84 , srv485299373:84 , srv528290806:84 , srv624641128:84 , srv673726989:84 , srv710979266:84 , srv762673198:84 , srv805264420:84 , srv819138755:84 , srv819546707:84 , srv949981234:84 , srv10580543:85 , srv1059282567:85 , srv1080589145:85 , srv1098354050:85 , srv111323899:85 , srv1122874351:85 , srv1138137831:85 , srv120320351:85 , srv1262112443:85 , srv1296747128:85 , srv1330223342:85 , srv1349299138:85 , srv135743265:85 , srv1382999493:85 , srv1438766086:85 , srv1441669762:85 , srv1446741366:85 , srv1456251033:85 , srv1466886555:85 , srv1491533591:85 , srv168920599:85 , srv1700223846:85 , srv1705109999:85 , srv1732007244:85 , srv1756485384:85 , srv1769229551:85 , srv1783952282:85 , srv1818746009:85 , srv1835432502:85 , srv186790685:85 , srv1900772460:85 , srv197196532:85 , srv205426767:85 , srv2056233397:85 , srv2119777377:85 , srv2141084081:85 , srv2142470779:85 , srv2145528765:85 , srv234333668:85 , srv268245727:85 , srv372527811:85 , srv513969371:85 , srv523907576:85 , srv525162889:85 , srv572867635:85 , srv579918724:85 , srv601086492:85 , srv618976154:85 , srv670531863:85 , srv67354313:85 , srv701583895:85 , srv729768264:85 , srv730211645:85 , srv731154348:85 , srv865314586:85 , srv596923983:86 , srv1078154944:636 }
2023-09-24 08:53:18,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:18,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1286363750=16, srv2036480623=54, srv407683054=72, srv2141084081=59, srv268245727=64, srv120320351=14, srv1873038188=46, srv819138755=96, srv2119777377=58, srv1059282567=3, srv513969371=75, srv670531863=85, srv197196532=53, srv528290806=78, srv2104519869=57, srv81087729=95, srv1035735470=0, srv1722977996=37, srv108918427=6, srv579918724=80, srv168920599=34, srv865314586=98, srv1330223342=18, srv372527811=66, srv1138137831=12, srv1818746009=43, srv1835432502=44, srv1080589145=5, srv1122874351=10, srv135743265=20, srv1098354050=8, srv2056233397=56, srv1456251033=25, srv1446741366=24, srv710979266=89, srv1480600183=29, srv1441669762=23, srv525162889=77, srv1555929688=31, srv205426767=55, srv485299373=74, srv618976154=83, srv1058193955=2, srv1926031882=48, srv186790685=45, srv1732007244=38, srv1669590542=33, srv40640022=71, srv596923983=81, srv1955888818=49, srv1757664417=40, srv572867635=79, srv1466886555=27, srv1637290291=32, srv731154348=92, srv1382999493=21, srv1970393871=52, srv1479529106=28, srv1900772460=47, srv1460897749=26, srv730211645=91, srv673726989=87, srv382166385=68, srv1349299138=19, srv2142470779=60, srv1078154944=4, srv624641128=84, srv373472940=67, srv67354313=86, srv1097961769=7, srv384972826=70, srv819546707=97, srv1769229551=41, srv337457506=65, srv2145528765=61, srv805264420=94, srv384439738=69, srv1262112443=15, srv1151742588=13, srv267315532=63, srv1961351483=51, srv701583895=88, srv10580543=1, srv1296747128=17, srv729768264=90, srv1756485384=39, srv1705109999=36, srv1783952282=42, srv949981234=99, srv112788601=11, srv1491533591=30, srv1438766086=22, srv1958978805=50, srv234333668=62, srv601086492=82, srv762673198=93, srv1700223846=35, srv523907576=76, srv111323899=9, srv447715617=73} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:18,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:18,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 08:53:18,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 08:53:18,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 08:53:18,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 08:53:18,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 08:53:18,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 08:53:18,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 08:53:18,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 08:53:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 08:53:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 08:53:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 08:53:18,446 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:18,446 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-24 08:53:18,447 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.4965234182829804E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018349396085439343); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-24 08:53:18,687 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 353 ms to try 5000 different iterations.  Found a solution that moves 21 regions; Going from a computed imbalance of 2.4965234182829804E-5 to a new imbalance of 6.8911141966908056E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0023333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:18,687 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1438766086:83 , srv1456251033:83 , srv197196532:83 , srv596923983:83 , srv710979266:83 , srv10580543:84 , srv1058193955:84 , srv1059282567:84 , srv1080589145:84 , srv108918427:84 , srv1097961769:84 , srv1098354050:84 , srv111323899:84 , srv1122874351:84 , srv112788601:84 , srv1138137831:84 , srv1151742588:84 , srv120320351:84 , srv1262112443:84 , srv1286363750:84 , srv1296747128:84 , srv1330223342:84 , srv1349299138:84 , srv135743265:84 , srv1382999493:84 , srv1441669762:84 , srv1446741366:84 , srv1460897749:84 , srv1466886555:84 , srv1479529106:84 , srv1480600183:84 , srv1491533591:84 , srv1555929688:84 , srv1637290291:84 , srv1669590542:84 , srv168920599:84 , srv1700223846:84 , srv1705109999:84 , srv1722977996:84 , srv1732007244:84 , srv1756485384:84 , srv1757664417:84 , srv1769229551:84 , srv1783952282:84 , srv1818746009:84 , srv1835432502:84 , srv186790685:84 , srv1873038188:84 , srv1900772460:84 , srv1955888818:84 , srv1958978805:84 , srv1961351483:84 , srv205426767:84 , srv2056233397:84 , srv2104519869:84 , srv2119777377:84 , srv2141084081:84 , srv2142470779:84 , srv2145528765:84 , srv234333668:84 , srv267315532:84 , srv268245727:84 , srv337457506:84 , srv372527811:84 , srv373472940:84 , srv382166385:84 , srv384439738:84 , srv384972826:84 , srv40640022:84 , srv407683054:84 , srv447715617:84 , srv485299373:84 , srv513969371:84 , srv523907576:84 , srv525162889:84 , srv528290806:84 , srv572867635:84 , srv579918724:84 , srv601086492:84 , srv618976154:84 , srv624641128:84 , srv670531863:84 , srv67354313:84 , srv673726989:84 , srv701583895:84 , srv729768264:84 , srv730211645:84 , srv731154348:84 , srv762673198:84 , srv805264420:84 , srv819138755:84 , srv819546707:84 , srv865314586:84 , srv949981234:84 , srv1035735470:85 , srv1926031882:85 , srv1970393871:85 , srv2036480623:85 , srv81087729:85 , srv1078154944:684 }
2023-09-24 08:53:18,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:18,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1286363750=16, srv2036480623=54, srv407683054=72, srv2141084081=59, srv268245727=64, srv120320351=14, srv1873038188=46, srv819138755=96, srv2119777377=58, srv1059282567=3, srv513969371=75, srv670531863=85, srv197196532=53, srv528290806=78, srv2104519869=57, srv81087729=95, srv1035735470=0, srv1722977996=37, srv108918427=6, srv579918724=80, srv168920599=34, srv865314586=98, srv1330223342=18, srv372527811=66, srv1138137831=12, srv1818746009=43, srv1835432502=44, srv1080589145=5, srv1122874351=10, srv135743265=20, srv1098354050=8, srv2056233397=56, srv1456251033=25, srv1446741366=24, srv710979266=89, srv1480600183=29, srv1441669762=23, srv525162889=77, srv1555929688=31, srv205426767=55, srv485299373=74, srv618976154=83, srv1058193955=2, srv1926031882=48, srv186790685=45, srv1732007244=38, srv1669590542=33, srv40640022=71, srv596923983=81, srv1955888818=49, srv1757664417=40, srv572867635=79, srv1466886555=27, srv1637290291=32, srv731154348=92, srv1382999493=21, srv1970393871=52, srv1479529106=28, srv1900772460=47, srv1460897749=26, srv730211645=91, srv673726989=87, srv382166385=68, srv1349299138=19, srv2142470779=60, srv1078154944=4, srv624641128=84, srv373472940=67, srv67354313=86, srv1097961769=7, srv384972826=70, srv819546707=97, srv1769229551=41, srv337457506=65, srv2145528765=61, srv805264420=94, srv384439738=69, srv1262112443=15, srv1151742588=13, srv267315532=63, srv1961351483=51, srv701583895=88, srv10580543=1, srv1296747128=17, srv729768264=90, srv1756485384=39, srv1705109999=36, srv1783952282=42, srv949981234=99, srv112788601=11, srv1491533591=30, srv1438766086=22, srv1958978805=50, srv234333668=62, srv601086492=82, srv762673198=93, srv1700223846=35, srv523907576=76, srv111323899=9, srv447715617=73} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:18,703 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:18,704 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-24 08:53:18,705 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-24 08:53:18,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-24 08:53:18,707 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-24 08:53:18,708 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-24 08:53:18,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-24 08:53:18,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-24 08:53:18,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-24 08:53:18,709 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-24 08:53:18,831 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:18,832 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1438766086:83 , srv1456251033:83 , srv197196532:83 , srv596923983:83 , srv710979266:83 , srv10580543:84 , srv1058193955:84 , srv1059282567:84 , srv1080589145:84 , srv108918427:84 , srv1097961769:84 , srv1098354050:84 , srv111323899:84 , srv1122874351:84 , srv112788601:84 , srv1138137831:84 , srv1151742588:84 , srv120320351:84 , srv1262112443:84 , srv1286363750:84 , srv1296747128:84 , srv1330223342:84 , srv1349299138:84 , srv135743265:84 , srv1382999493:84 , srv1441669762:84 , srv1446741366:84 , srv1460897749:84 , srv1466886555:84 , srv1479529106:84 , srv1480600183:84 , srv1491533591:84 , srv1555929688:84 , srv1637290291:84 , srv1669590542:84 , srv168920599:84 , srv1700223846:84 , srv1705109999:84 , srv1722977996:84 , srv1732007244:84 , srv1756485384:84 , srv1757664417:84 , srv1769229551:84 , srv1783952282:84 , srv1818746009:84 , srv1835432502:84 , srv186790685:84 , srv1873038188:84 , srv1900772460:84 , srv1955888818:84 , srv1958978805:84 , srv1961351483:84 , srv205426767:84 , srv2056233397:84 , srv2104519869:84 , srv2119777377:84 , srv2141084081:84 , srv2142470779:84 , srv2145528765:84 , srv234333668:84 , srv267315532:84 , srv268245727:84 , srv337457506:84 , srv372527811:84 , srv373472940:84 , srv382166385:84 , srv384439738:84 , srv384972826:84 , srv40640022:84 , srv407683054:84 , srv447715617:84 , srv485299373:84 , srv513969371:84 , srv523907576:84 , srv525162889:84 , srv528290806:84 , srv572867635:84 , srv579918724:84 , srv601086492:84 , srv618976154:84 , srv624641128:84 , srv670531863:84 , srv67354313:84 , srv673726989:84 , srv701583895:84 , srv729768264:84 , srv730211645:84 , srv731154348:84 , srv762673198:84 , srv805264420:84 , srv819138755:84 , srv819546707:84 , srv865314586:84 , srv949981234:84 , srv1035735470:85 , srv1926031882:85 , srv1970393871:85 , srv2036480623:85 , srv81087729:85 , srv1078154944:684 }
2023-09-24 08:53:18,859 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=314 (was 314), ProcessCount=171 (was 171), AvailableMemoryMB=9471 (was 10109)
2023-09-24 08:53:18,867 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=171, AvailableMemoryMB=9471
2023-09-24 08:53:18,867 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-24 08:53:18,868 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-24 08:53:18,879 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006825313:84 , srv103264210:84 , srv1154625282:84 , srv1326416463:84 , srv1638008117:84 , srv1647278852:84 , srv1746704863:84 , srv1812361798:84 , srv1828873572:84 , srv1893583720:84 , srv1907095196:84 , srv2090107121:84 , srv2091218028:84 , srv309004756:84 , srv374698946:84 , srv392674243:84 , srv559827107:84 , srv576139520:84 , srv630970017:84 , srv639242445:84 , srv654633569:84 , srv681282848:84 , srv736599909:84 , srv757851311:84 , srv871400800:84 , srv896445361:84 , srv961128651:84 , srv961658704:84 , srv976379459:84 , srv742087655:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-24 08:53:18,880 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:53:18,881 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1647278852=5, srv309004756=13, srv896445361=26, srv1828873572=8, srv374698946=14, srv630970017=18, srv757851311=24, srv742087655=23, srv576139520=17, srv961128651=27, srv639242445=19, srv736599909=22, srv871400800=25, srv1638008117=4, srv1006825313=0, srv1154625282=2, srv2091218028=12, srv559827107=16, srv681282848=21, srv961658704=28, srv1907095196=10, srv103264210=1, srv1326416463=3, srv1746704863=6, srv976379459=29, srv392674243=15, srv1812361798=7, srv1893583720=9, srv2090107121=11, srv654633569=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:53:18,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:53:18,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-24 08:53:18,910 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.122407672591855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:53:18,910 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912578745843639, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.122407672591855, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-24 08:55:18,896 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5536273 different iterations.  Found a solution that moves 1875 regions; Going from a computed imbalance of 0.9912578745843639 to a new imbalance of 4.376343537466002E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6944444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:55:18,903 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1006825313:90 , srv103264210:90 , srv1154625282:90 , srv1326416463:90 , srv1638008117:90 , srv1647278852:90 , srv1746704863:90 , srv1812361798:90 , srv1828873572:90 , srv1893583720:90 , srv1907095196:90 , srv2090107121:90 , srv2091218028:90 , srv309004756:90 , srv374698946:90 , srv392674243:90 , srv559827107:90 , srv576139520:90 , srv630970017:90 , srv639242445:90 , srv654633569:90 , srv681282848:90 , srv736599909:90 , srv742087655:90 , srv757851311:90 , srv871400800:90 , srv896445361:90 , srv961128651:90 , srv961658704:90 , srv976379459:90 }
2023-09-24 08:55:18,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-24 08:55:18,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1647278852=5, srv309004756=13, srv896445361=26, srv1828873572=8, srv374698946=14, srv630970017=18, srv757851311=24, srv742087655=23, srv576139520=17, srv961128651=27, srv639242445=19, srv736599909=22, srv871400800=25, srv1638008117=4, srv1006825313=0, srv1154625282=2, srv2091218028=12, srv559827107=16, srv681282848=21, srv961658704=28, srv1907095196=10, srv103264210=1, srv1326416463=3, srv1746704863=6, srv976379459=29, srv392674243=15, srv1812361798=7, srv1893583720=9, srv2090107121=11, srv654633569=20} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-24 08:55:18,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-24 08:55:18,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-24 08:55:18,929 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-24 08:55:18,929 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1006825313:90 , srv103264210:90 , srv1154625282:90 , srv1326416463:90 , srv1638008117:90 , srv1647278852:90 , srv1746704863:90 , srv1812361798:90 , srv1828873572:90 , srv1893583720:90 , srv1907095196:90 , srv2090107121:90 , srv2091218028:90 , srv309004756:90 , srv374698946:90 , srv392674243:90 , srv559827107:90 , srv576139520:90 , srv630970017:90 , srv639242445:90 , srv654633569:90 , srv681282848:90 , srv736599909:90 , srv742087655:90 , srv757851311:90 , srv871400800:90 , srv896445361:90 , srv961128651:90 , srv961658704:90 , srv976379459:90 }
2023-09-24 08:55:18,945 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=176 (was 314), ProcessCount=169 (was 171), AvailableMemoryMB=8251 (was 9471)