2023-03-23 19:56:41,056 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-23 19:56:41,369 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-23 19:56:41,471 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-23 19:56:41,526 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 19:56:41,533 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-23 19:56:41,545 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=279, ProcessCount=169, AvailableMemoryMB=6421
2023-03-23 19:56:41,547 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 19:56:41,548 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-23 19:56:41,661 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1053292593:84 , srv1086218488:84 , srv1088787590:84 , srv112268727:84 , srv1150511117:84 , srv1197842738:84 , srv1219707750:84 , srv1267613118:84 , srv1280134348:84 , srv1285583168:84 , srv1302963508:84 , srv1311417662:84 , srv132371085:84 , srv1338470390:84 , srv1365617899:84 , srv1372786654:84 , srv1379269477:84 , srv1381524239:84 , srv1404237313:84 , srv1417273187:84 , srv1438195679:84 , srv1459257619:84 , srv1492382957:84 , srv1518401607:84 , srv1530835058:84 , srv1532086264:84 , srv1554198740:84 , srv1558692040:84 , srv1561761590:84 , srv1565893944:84 , srv1569068186:84 , srv159542064:84 , srv163983122:84 , srv1674674354:84 , srv1691068595:84 , srv1712536335:84 , srv1725480315:84 , srv1729318174:84 , srv1796905425:84 , srv1810428386:84 , srv1851910113:84 , srv1870008745:84 , srv187433042:84 , srv1883781267:84 , srv1892930211:84 , srv1922334992:84 , srv193547147:84 , srv1940115340:84 , srv1944676537:84 , srv1967373712:84 , srv1980071618:84 , srv1992438417:84 , srv1993627597:84 , srv2040649273:84 , srv2045650595:84 , srv2060317188:84 , srv2069770356:84 , srv2083094725:84 , srv2127396611:84 , srv2138256600:84 , srv236054127:84 , srv266679266:84 , srv274293460:84 , srv279011155:84 , srv283435890:84 , srv321704309:84 , srv360716145:84 , srv394369026:84 , srv406728476:84 , srv49102239:84 , srv497029728:84 , srv540930761:84 , srv541409149:84 , srv584703701:84 , srv667548409:84 , srv680053390:84 , srv682159499:84 , srv74939033:84 , srv757872266:84 , srv764179246:84 , srv765638696:84 , srv791992119:84 , srv804815127:84 , srv813629429:84 , srv83007388:84 , srv840529212:84 , srv863619895:84 , srv871226566:84 , srv880603490:84 , srv88080373:84 , srv891111007:84 , srv929219329:84 , srv93048731:84 , srv931758159:84 , srv93925314:84 , srv950508556:84 , srv967895003:84 , srv976518241:84 , srv986730609:84 , srv1611529895:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-23 19:56:41,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:56:41,675 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1285583168=9, srv1438195679=20, srv541409149=73, srv667548409=75, srv1280134348=8, srv540930761=72, srv863619895=87, srv1150511117=4, srv791992119=82, srv765638696=81, srv840529212=86, srv1518401607=23, srv1379269477=16, srv1267613118=7, srv757872266=79, srv406728476=69, srv950508556=96, srv1611529895=32, srv1691068595=35, srv1554198740=26, srv986730609=99, srv2045650595=55, srv83007388=85, srv112268727=3, srv1565893944=29, srv813629429=84, srv283435890=65, srv1674674354=34, srv274293460=63, srv49102239=70, srv680053390=76, srv394369026=68, srv497029728=71, srv1492382957=22, srv1561761590=28, srv132371085=12, srv1940115340=48, srv2060317188=56, srv2083094725=58, srv1729318174=38, srv871226566=88, srv236054127=61, srv93925314=95, srv1532086264=25, srv1338470390=13, srv187433042=43, srv1967373712=50, srv266679266=62, srv1459257619=21, srv159542064=31, srv279011155=64, srv1404237313=18, srv1883781267=44, srv1311417662=11, srv1053292593=0, srv1219707750=6, srv584703701=74, srv1993627597=53, srv1796905425=39, srv74939033=78, srv1870008745=42, srv891111007=91, srv1810428386=40, srv193547147=47, srv88080373=90, srv804815127=83, srv163983122=33, srv931758159=94, srv1712536335=36, srv1992438417=52, srv1086218488=1, srv1381524239=17, srv976518241=98, srv1088787590=2, srv1569068186=30, srv1372786654=15, srv2127396611=59, srv1980071618=51, srv1558692040=27, srv880603490=89, srv1851910113=41, srv929219329=92, srv1922334992=46, srv360716145=67, srv1892930211=45, srv1530835058=24, srv764179246=80, srv93048731=93, srv2040649273=54, srv682159499=77, srv1302963508=10, srv2069770356=57, srv1944676537=49, srv321704309=66, srv1365617899=14, srv1417273187=19, srv1725480315=37, srv1197842738=5, srv2138256600=60, srv967895003=97} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:56:41,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:56:41,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:56:41,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 19:56:41,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 19:56:41,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 19:56:41,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 19:56:41,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 19:56:41,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 19:56:41,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 19:56:41,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 19:56:41,967 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:41,968 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 19:56:41,968 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 19:56:42,285 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 482 ms to try 5000 different iterations.  Found a solution that moves 4715 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17561656622844787. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04126567238844572, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05576727334786455, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5238888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04126567238844572, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.242, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:42,296 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1311417662:82 , srv163983122:82 , srv1302963508:83 , srv193547147:83 , srv1967373712:83 , srv274293460:83 , srv283435890:83 , srv360716145:83 , srv840529212:83 , srv931758159:83 , srv112268727:84 , srv1285583168:84 , srv1674674354:84 , srv1691068595:84 , srv1870008745:84 , srv2060317188:84 , srv2127396611:84 , srv266679266:84 , srv680053390:84 , srv74939033:84 , srv804815127:84 , srv929219329:84 , srv1086218488:85 , srv1088787590:85 , srv1267613118:85 , srv1338470390:85 , srv1365617899:85 , srv1372786654:85 , srv1381524239:85 , srv1561761590:85 , srv1565893944:85 , srv2040649273:85 , srv2045650595:85 , srv394369026:85 , srv497029728:85 , srv757872266:85 , srv765638696:85 , srv93925314:85 , srv1053292593:86 , srv1197842738:86 , srv1404237313:86 , srv1417273187:86 , srv1518401607:86 , srv1569068186:86 , srv159542064:86 , srv1796905425:86 , srv1810428386:86 , srv1922334992:86 , srv1940115340:86 , srv1944676537:86 , srv1980071618:86 , srv2069770356:86 , srv2083094725:86 , srv764179246:86 , srv791992119:86 , srv88080373:86 , srv93048731:86 , srv1150511117:87 , srv1492382957:87 , srv1554198740:87 , srv1558692040:87 , srv1712536335:87 , srv1993627597:87 , srv279011155:87 , srv406728476:87 , srv49102239:87 , srv540930761:87 , srv541409149:87 , srv667548409:87 , srv967895003:87 , srv976518241:87 , srv1280134348:88 , srv132371085:88 , srv1438195679:88 , srv1729318174:88 , srv187433042:88 , srv1883781267:88 , srv1892930211:88 , srv2138256600:88 , srv236054127:88 , srv584703701:88 , srv682159499:88 , srv863619895:88 , srv891111007:88 , srv986730609:88 , srv1379269477:89 , srv1459257619:89 , srv1725480315:89 , srv1851910113:89 , srv1992438417:89 , srv83007388:89 , srv321704309:90 , srv871226566:90 , srv950508556:90 , srv1219707750:91 , srv1532086264:91 , srv813629429:92 , srv880603490:92 , srv1530835058:93 , srv1611529895:457 }
2023-03-23 19:56:42,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:56:42,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1285583168=9, srv1438195679=20, srv541409149=73, srv667548409=75, srv1280134348=8, srv540930761=72, srv863619895=87, srv1150511117=4, srv791992119=82, srv765638696=81, srv840529212=86, srv1518401607=23, srv1379269477=16, srv1267613118=7, srv757872266=79, srv406728476=69, srv950508556=96, srv1611529895=32, srv1691068595=35, srv1554198740=26, srv986730609=99, srv2045650595=55, srv83007388=85, srv112268727=3, srv1565893944=29, srv813629429=84, srv283435890=65, srv1674674354=34, srv274293460=63, srv49102239=70, srv680053390=76, srv394369026=68, srv497029728=71, srv1492382957=22, srv1561761590=28, srv132371085=12, srv1940115340=48, srv2060317188=56, srv2083094725=58, srv1729318174=38, srv871226566=88, srv236054127=61, srv93925314=95, srv1532086264=25, srv1338470390=13, srv187433042=43, srv1967373712=50, srv266679266=62, srv1459257619=21, srv159542064=31, srv279011155=64, srv1404237313=18, srv1883781267=44, srv1311417662=11, srv1053292593=0, srv1219707750=6, srv584703701=74, srv1993627597=53, srv1796905425=39, srv74939033=78, srv1870008745=42, srv891111007=91, srv1810428386=40, srv193547147=47, srv88080373=90, srv804815127=83, srv163983122=33, srv931758159=94, srv1712536335=36, srv1992438417=52, srv1086218488=1, srv1381524239=17, srv976518241=98, srv1088787590=2, srv1569068186=30, srv1372786654=15, srv2127396611=59, srv1980071618=51, srv1558692040=27, srv880603490=89, srv1851910113=41, srv929219329=92, srv1922334992=46, srv360716145=67, srv1892930211=45, srv1530835058=24, srv764179246=80, srv93048731=93, srv2040649273=54, srv682159499=77, srv1302963508=10, srv2069770356=57, srv1944676537=49, srv321704309=66, srv1365617899=14, srv1417273187=19, srv1725480315=37, srv1197842738=5, srv2138256600=60, srv967895003=97} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:56:42,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:56:42,316 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:56:42,317 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:56:42,318 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 19:56:42,319 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 19:56:42,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 19:56:42,321 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 19:56:42,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 19:56:42,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 19:56:42,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 19:56:42,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 19:56:42,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 19:56:42,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 19:56:42,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 19:56:42,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 19:56:42,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 19:56:42,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 19:56:42,452 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04126567238844572, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05576727334786455, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04126567238844572, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.242, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:42,453 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 19:56:42,453 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17558355109280124, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04126567238844572, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05576727334786455, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04126567238844572, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17033333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.242, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 19:56:42,754 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 409 ms to try 5000 different iterations.  Found a solution that moves 2638 regions; Going from a computed imbalance of 0.17558355109280124 to a new imbalance of 0.01386885972017273. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022377682930841743); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0363083193170296, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2931111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022377682930841743); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:42,763 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1530835058:78 , srv1532086264:79 , srv880603490:79 , srv321704309:81 , srv871226566:81 , srv950508556:81 , srv1219707750:82 , srv1379269477:82 , srv1438195679:82 , srv1459257619:82 , srv1883781267:82 , srv1725480315:83 , srv1729318174:83 , srv1892930211:83 , srv1992438417:83 , srv584703701:83 , srv667548409:83 , srv813629429:83 , srv132371085:84 , srv1554198740:84 , srv1558692040:84 , srv1712536335:84 , srv187433042:84 , srv1940115340:84 , srv540930761:84 , srv541409149:84 , srv682159499:84 , srv83007388:84 , srv891111007:84 , srv986730609:84 , srv1053292593:85 , srv112268727:85 , srv1197842738:85 , srv1267613118:85 , srv1365617899:85 , srv1404237313:85 , srv1417273187:85 , srv1851910113:85 , srv1980071618:85 , srv2083094725:85 , srv236054127:85 , srv49102239:85 , srv791992119:85 , srv863619895:85 , srv88080373:85 , srv93925314:85 , srv976518241:85 , srv1086218488:86 , srv1088787590:86 , srv1150511117:86 , srv1492382957:86 , srv1518401607:86 , srv1922334992:86 , srv1993627597:86 , srv2045650595:86 , srv2138256600:86 , srv279011155:86 , srv406728476:86 , srv757872266:86 , srv967895003:86 , srv1280134348:87 , srv1285583168:87 , srv1338470390:87 , srv1569068186:87 , srv159542064:87 , srv1796905425:87 , srv1810428386:87 , srv1944676537:87 , srv2060317188:87 , srv2069770356:87 , srv2127396611:87 , srv765638696:87 , srv1381524239:88 , srv1561761590:88 , srv1565893944:88 , srv1870008745:88 , srv2040649273:88 , srv266679266:88 , srv497029728:88 , srv74939033:88 , srv764179246:88 , srv804815127:88 , srv840529212:88 , srv929219329:88 , srv93048731:88 , srv1302963508:89 , srv1372786654:89 , srv1691068595:89 , srv274293460:89 , srv360716145:89 , srv394369026:89 , srv931758159:89 , srv1311417662:90 , srv163983122:90 , srv193547147:90 , srv1967373712:90 , srv680053390:90 , srv1674674354:91 , srv283435890:91 , srv1611529895:516 }
2023-03-23 19:56:42,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:56:42,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1285583168=9, srv1438195679=20, srv541409149=73, srv667548409=75, srv1280134348=8, srv540930761=72, srv863619895=87, srv1150511117=4, srv791992119=82, srv765638696=81, srv840529212=86, srv1518401607=23, srv1379269477=16, srv1267613118=7, srv757872266=79, srv406728476=69, srv950508556=96, srv1611529895=32, srv1691068595=35, srv1554198740=26, srv986730609=99, srv2045650595=55, srv83007388=85, srv112268727=3, srv1565893944=29, srv813629429=84, srv283435890=65, srv1674674354=34, srv274293460=63, srv49102239=70, srv680053390=76, srv394369026=68, srv497029728=71, srv1492382957=22, srv1561761590=28, srv132371085=12, srv1940115340=48, srv2060317188=56, srv2083094725=58, srv1729318174=38, srv871226566=88, srv236054127=61, srv93925314=95, srv1532086264=25, srv1338470390=13, srv187433042=43, srv1967373712=50, srv266679266=62, srv1459257619=21, srv159542064=31, srv279011155=64, srv1404237313=18, srv1883781267=44, srv1311417662=11, srv1053292593=0, srv1219707750=6, srv584703701=74, srv1993627597=53, srv1796905425=39, srv74939033=78, srv1870008745=42, srv891111007=91, srv1810428386=40, srv193547147=47, srv88080373=90, srv804815127=83, srv163983122=33, srv931758159=94, srv1712536335=36, srv1992438417=52, srv1086218488=1, srv1381524239=17, srv976518241=98, srv1088787590=2, srv1569068186=30, srv1372786654=15, srv2127396611=59, srv1980071618=51, srv1558692040=27, srv880603490=89, srv1851910113=41, srv929219329=92, srv1922334992=46, srv360716145=67, srv1892930211=45, srv1530835058=24, srv764179246=80, srv93048731=93, srv2040649273=54, srv682159499=77, srv1302963508=10, srv2069770356=57, srv1944676537=49, srv321704309=66, srv1365617899=14, srv1417273187=19, srv1725480315=37, srv1197842738=5, srv2138256600=60, srv967895003=97} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:56:42,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:56:42,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:56:42,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:56:42,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:56:42,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:56:42,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:56:42,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 19:56:42,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 19:56:42,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 19:56:42,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 19:56:42,784 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 19:56:42,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 19:56:42,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 19:56:42,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 19:56:42,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 19:56:42,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022377682930841743); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0363083193170296, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022377682930841743); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:42,893 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 19:56:42,893 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013850388049369793, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022377682930841743); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0363083193170296, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022377682930841743); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010916666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041666666666666664, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 19:56:43,182 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 384 ms to try 5000 different iterations.  Found a solution that moves 933 regions; Going from a computed imbalance of 0.013850388049369793 to a new imbalance of 7.718338419822122E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008804311428960357); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01059402876939547); MoveCostFunction : (multiplier=7.0, imbalance=0.10366666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008804311428960357); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:43,186 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1280134348:82 , srv1674674354:82 , srv394369026:82 , srv813629429:82 , srv1219707750:83 , srv1381524239:83 , srv159542064:83 , srv1691068595:83 , srv193547147:83 , srv283435890:83 , srv497029728:83 , srv764179246:83 , srv93048731:83 , srv1150511117:84 , srv132371085:84 , srv1372786654:84 , srv1561761590:84 , srv1565893944:84 , srv1569068186:84 , srv163983122:84 , srv1796905425:84 , srv1810428386:84 , srv1851910113:84 , srv1944676537:84 , srv1967373712:84 , srv1993627597:84 , srv2040649273:84 , srv2069770356:84 , srv266679266:84 , srv360716145:84 , srv680053390:84 , srv967895003:84 , srv1088787590:85 , srv1285583168:85 , srv1338470390:85 , srv1417273187:85 , srv1492382957:85 , srv1870008745:85 , srv187433042:85 , srv1892930211:85 , srv1992438417:85 , srv2127396611:85 , srv2138256600:85 , srv274293460:85 , srv406728476:85 , srv49102239:85 , srv74939033:85 , srv765638696:85 , srv804815127:85 , srv83007388:85 , srv863619895:85 , srv891111007:85 , srv929219329:85 , srv931758159:85 , srv976518241:85 , srv1086218488:86 , srv1302963508:86 , srv1311417662:86 , srv1365617899:86 , srv1404237313:86 , srv1459257619:86 , srv1518401607:86 , srv1530835058:86 , srv1554198740:86 , srv1558692040:86 , srv1712536335:86 , srv1725480315:86 , srv1922334992:86 , srv1980071618:86 , srv2045650595:86 , srv2083094725:86 , srv236054127:86 , srv279011155:86 , srv321704309:86 , srv682159499:86 , srv757872266:86 , srv871226566:86 , srv880603490:86 , srv93925314:86 , srv1053292593:87 , srv112268727:87 , srv1197842738:87 , srv1532086264:87 , srv1729318174:87 , srv1940115340:87 , srv2060317188:87 , srv540930761:87 , srv541409149:87 , srv584703701:87 , srv791992119:87 , srv88080373:87 , srv950508556:87 , srv986730609:87 , srv1267613118:88 , srv1438195679:88 , srv1883781267:88 , srv667548409:88 , srv840529212:88 , srv1379269477:89 , srv1611529895:563 }
2023-03-23 19:56:43,189 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:56:43,190 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1285583168=9, srv1438195679=20, srv541409149=73, srv667548409=75, srv1280134348=8, srv540930761=72, srv863619895=87, srv1150511117=4, srv791992119=82, srv765638696=81, srv840529212=86, srv1518401607=23, srv1379269477=16, srv1267613118=7, srv757872266=79, srv406728476=69, srv950508556=96, srv1611529895=32, srv1691068595=35, srv1554198740=26, srv986730609=99, srv2045650595=55, srv83007388=85, srv112268727=3, srv1565893944=29, srv813629429=84, srv283435890=65, srv1674674354=34, srv274293460=63, srv49102239=70, srv680053390=76, srv394369026=68, srv497029728=71, srv1492382957=22, srv1561761590=28, srv132371085=12, srv1940115340=48, srv2060317188=56, srv2083094725=58, srv1729318174=38, srv871226566=88, srv236054127=61, srv93925314=95, srv1532086264=25, srv1338470390=13, srv187433042=43, srv1967373712=50, srv266679266=62, srv1459257619=21, srv159542064=31, srv279011155=64, srv1404237313=18, srv1883781267=44, srv1311417662=11, srv1053292593=0, srv1219707750=6, srv584703701=74, srv1993627597=53, srv1796905425=39, srv74939033=78, srv1870008745=42, srv891111007=91, srv1810428386=40, srv193547147=47, srv88080373=90, srv804815127=83, srv163983122=33, srv931758159=94, srv1712536335=36, srv1992438417=52, srv1086218488=1, srv1381524239=17, srv976518241=98, srv1088787590=2, srv1569068186=30, srv1372786654=15, srv2127396611=59, srv1980071618=51, srv1558692040=27, srv880603490=89, srv1851910113=41, srv929219329=92, srv1922334992=46, srv360716145=67, srv1892930211=45, srv1530835058=24, srv764179246=80, srv93048731=93, srv2040649273=54, srv682159499=77, srv1302963508=10, srv2069770356=57, srv1944676537=49, srv321704309=66, srv1365617899=14, srv1417273187=19, srv1725480315=37, srv1197842738=5, srv2138256600=60, srv967895003=97} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:56:43,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:56:43,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:56:43,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:56:43,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:56:43,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:56:43,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 19:56:43,210 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 19:56:43,211 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 19:56:43,212 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 19:56:43,213 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 19:56:43,214 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 19:56:43,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 19:56:43,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 19:56:43,215 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 19:56:43,332 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008804311428960357); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01059402876939547); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008804311428960357); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:43,332 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-23 19:56:43,332 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.653008363494831E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008804311428960357); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01059402876939547); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.008804311428960357); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-23 19:56:43,612 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 380 ms to try 5000 different iterations.  Found a solution that moves 315 regions; Going from a computed imbalance of 7.653008363494831E-4 to a new imbalance of 9.642000775949281E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.035, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:43,614 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1379269477:82 , srv279011155:82 , srv1267613118:83 , srv2138256600:83 , srv236054127:83 , srv680053390:83 , srv840529212:83 , srv1053292593:84 , srv1150511117:84 , srv1197842738:84 , srv1280134348:84 , srv1302963508:84 , srv1311417662:84 , srv1372786654:84 , srv1438195679:84 , srv1492382957:84 , srv1518401607:84 , srv1561761590:84 , srv1712536335:84 , srv1725480315:84 , srv1729318174:84 , srv1851910113:84 , srv1883781267:84 , srv1922334992:84 , srv2060317188:84 , srv406728476:84 , srv540930761:84 , srv541409149:84 , srv584703701:84 , srv682159499:84 , srv791992119:84 , srv83007388:84 , srv863619895:84 , srv950508556:84 , srv986730609:84 , srv1086218488:85 , srv1285583168:85 , srv1338470390:85 , srv1404237313:85 , srv1530835058:85 , srv1532086264:85 , srv1554198740:85 , srv1558692040:85 , srv1565893944:85 , srv1569068186:85 , srv1674674354:85 , srv1796905425:85 , srv1810428386:85 , srv1870008745:85 , srv187433042:85 , srv1940115340:85 , srv1944676537:85 , srv1967373712:85 , srv1980071618:85 , srv1992438417:85 , srv1993627597:85 , srv2040649273:85 , srv2045650595:85 , srv2069770356:85 , srv2083094725:85 , srv274293460:85 , srv321704309:85 , srv49102239:85 , srv667548409:85 , srv74939033:85 , srv757872266:85 , srv764179246:85 , srv765638696:85 , srv804815127:85 , srv813629429:85 , srv871226566:85 , srv88080373:85 , srv891111007:85 , srv929219329:85 , srv93048731:85 , srv931758159:85 , srv967895003:85 , srv976518241:85 , srv1088787590:86 , srv112268727:86 , srv1219707750:86 , srv132371085:86 , srv1365617899:86 , srv1381524239:86 , srv1417273187:86 , srv1459257619:86 , srv159542064:86 , srv1691068595:86 , srv1892930211:86 , srv193547147:86 , srv2127396611:86 , srv266679266:86 , srv283435890:86 , srv360716145:86 , srv394369026:86 , srv497029728:86 , srv880603490:86 , srv93925314:86 , srv163983122:87 , srv1611529895:607 }
2023-03-23 19:56:43,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:56:43,617 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1285583168=9, srv1438195679=20, srv541409149=73, srv667548409=75, srv1280134348=8, srv540930761=72, srv863619895=87, srv1150511117=4, srv791992119=82, srv765638696=81, srv840529212=86, srv1518401607=23, srv1379269477=16, srv1267613118=7, srv757872266=79, srv406728476=69, srv950508556=96, srv1611529895=32, srv1691068595=35, srv1554198740=26, srv986730609=99, srv2045650595=55, srv83007388=85, srv112268727=3, srv1565893944=29, srv813629429=84, srv283435890=65, srv1674674354=34, srv274293460=63, srv49102239=70, srv680053390=76, srv394369026=68, srv497029728=71, srv1492382957=22, srv1561761590=28, srv132371085=12, srv1940115340=48, srv2060317188=56, srv2083094725=58, srv1729318174=38, srv871226566=88, srv236054127=61, srv93925314=95, srv1532086264=25, srv1338470390=13, srv187433042=43, srv1967373712=50, srv266679266=62, srv1459257619=21, srv159542064=31, srv279011155=64, srv1404237313=18, srv1883781267=44, srv1311417662=11, srv1053292593=0, srv1219707750=6, srv584703701=74, srv1993627597=53, srv1796905425=39, srv74939033=78, srv1870008745=42, srv891111007=91, srv1810428386=40, srv193547147=47, srv88080373=90, srv804815127=83, srv163983122=33, srv931758159=94, srv1712536335=36, srv1992438417=52, srv1086218488=1, srv1381524239=17, srv976518241=98, srv1088787590=2, srv1569068186=30, srv1372786654=15, srv2127396611=59, srv1980071618=51, srv1558692040=27, srv880603490=89, srv1851910113=41, srv929219329=92, srv1922334992=46, srv360716145=67, srv1892930211=45, srv1530835058=24, srv764179246=80, srv93048731=93, srv2040649273=54, srv682159499=77, srv1302963508=10, srv2069770356=57, srv1944676537=49, srv321704309=66, srv1365617899=14, srv1417273187=19, srv1725480315=37, srv1197842738=5, srv2138256600=60, srv967895003=97} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:56:43,628 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:56:43,629 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-23 19:56:43,630 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-23 19:56:43,631 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-23 19:56:43,632 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-23 19:56:43,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-23 19:56:43,634 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-23 19:56:43,737 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.421433061660995E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:43,737 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1379269477:82 , srv279011155:82 , srv1267613118:83 , srv2138256600:83 , srv236054127:83 , srv680053390:83 , srv840529212:83 , srv1053292593:84 , srv1150511117:84 , srv1197842738:84 , srv1280134348:84 , srv1302963508:84 , srv1311417662:84 , srv1372786654:84 , srv1438195679:84 , srv1492382957:84 , srv1518401607:84 , srv1561761590:84 , srv1712536335:84 , srv1725480315:84 , srv1729318174:84 , srv1851910113:84 , srv1883781267:84 , srv1922334992:84 , srv2060317188:84 , srv406728476:84 , srv540930761:84 , srv541409149:84 , srv584703701:84 , srv682159499:84 , srv791992119:84 , srv83007388:84 , srv863619895:84 , srv950508556:84 , srv986730609:84 , srv1086218488:85 , srv1285583168:85 , srv1338470390:85 , srv1404237313:85 , srv1530835058:85 , srv1532086264:85 , srv1554198740:85 , srv1558692040:85 , srv1565893944:85 , srv1569068186:85 , srv1674674354:85 , srv1796905425:85 , srv1810428386:85 , srv1870008745:85 , srv187433042:85 , srv1940115340:85 , srv1944676537:85 , srv1967373712:85 , srv1980071618:85 , srv1992438417:85 , srv1993627597:85 , srv2040649273:85 , srv2045650595:85 , srv2069770356:85 , srv2083094725:85 , srv274293460:85 , srv321704309:85 , srv49102239:85 , srv667548409:85 , srv74939033:85 , srv757872266:85 , srv764179246:85 , srv765638696:85 , srv804815127:85 , srv813629429:85 , srv871226566:85 , srv88080373:85 , srv891111007:85 , srv929219329:85 , srv93048731:85 , srv931758159:85 , srv967895003:85 , srv976518241:85 , srv1088787590:86 , srv112268727:86 , srv1219707750:86 , srv132371085:86 , srv1365617899:86 , srv1381524239:86 , srv1417273187:86 , srv1459257619:86 , srv159542064:86 , srv1691068595:86 , srv1892930211:86 , srv193547147:86 , srv2127396611:86 , srv266679266:86 , srv283435890:86 , srv360716145:86 , srv394369026:86 , srv497029728:86 , srv880603490:86 , srv93925314:86 , srv163983122:87 , srv1611529895:607 }
2023-03-23 19:56:43,757 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=279 (was 279), ProcessCount=169 (was 169), AvailableMemoryMB=5931 (was 6421)
2023-03-23 19:56:43,765 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=279, ProcessCount=169, AvailableMemoryMB=5931
2023-03-23 19:56:43,766 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-23 19:56:43,766 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-23 19:56:43,773 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1089098131:84 , srv1130237638:84 , srv1174947418:84 , srv1316945850:84 , srv1336479784:84 , srv1424479901:84 , srv1479243937:84 , srv1509270064:84 , srv1521548240:84 , srv1642903489:84 , srv1656220104:84 , srv1703865817:84 , srv1802402542:84 , srv1816814887:84 , srv1871232203:84 , srv1994705263:84 , srv2002136700:84 , srv2033681713:84 , srv2090990781:84 , srv2116692923:84 , srv2120809637:84 , srv225372216:84 , srv363309005:84 , srv390744273:84 , srv453844452:84 , srv653220813:84 , srv662958393:84 , srv70996758:84 , srv913901453:84 , srv1369475543:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-23 19:56:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:56:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv225372216=22, srv1994705263=16, srv1336479784=4, srv2033681713=18, srv1642903489=10, srv1089098131=0, srv1479243937=7, srv1802402542=13, srv1369475543=5, srv1816814887=14, srv390744273=24, srv363309005=23, srv70996758=28, srv2002136700=17, srv1509270064=8, srv1521548240=9, srv1316945850=3, srv662958393=27, srv2120809637=21, srv913901453=29, srv1871232203=15, srv1130237638=1, srv453844452=25, srv1424479901=6, srv653220813=26, srv2090990781=19, srv1174947418=2, srv1656220104=11, srv2116692923=20, srv1703865817=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:56:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:56:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:56:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:56:43,783 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-23 19:56:43,796 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0933609239483668, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:56:43,797 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912487220487127, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0933609239483668, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-23 19:58:43,786 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6095678 different iterations.  Found a solution that moves 1858 regions; Going from a computed imbalance of 0.9912487220487127 to a new imbalance of 4.336664689392977E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6881481481481482, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:58:43,791 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1089098131:90 , srv1130237638:90 , srv1174947418:90 , srv1316945850:90 , srv1336479784:90 , srv1369475543:90 , srv1424479901:90 , srv1479243937:90 , srv1509270064:90 , srv1521548240:90 , srv1642903489:90 , srv1656220104:90 , srv1703865817:90 , srv1802402542:90 , srv1816814887:90 , srv1871232203:90 , srv1994705263:90 , srv2002136700:90 , srv2033681713:90 , srv2090990781:90 , srv2116692923:90 , srv2120809637:90 , srv225372216:90 , srv363309005:90 , srv390744273:90 , srv453844452:90 , srv653220813:90 , srv662958393:90 , srv70996758:90 , srv913901453:90 }
2023-03-23 19:58:43,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-23 19:58:43,793 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv225372216=22, srv1994705263=16, srv1336479784=4, srv2033681713=18, srv1642903489=10, srv1089098131=0, srv1479243937=7, srv1802402542=13, srv1369475543=5, srv1816814887=14, srv390744273=24, srv363309005=23, srv70996758=28, srv2002136700=17, srv1509270064=8, srv1521548240=9, srv1316945850=3, srv662958393=27, srv2120809637=21, srv913901453=29, srv1871232203=15, srv1130237638=1, srv453844452=25, srv1424479901=6, srv653220813=26, srv2090990781=19, srv1174947418=2, srv1656220104=11, srv2116692923=20, srv1703865817=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-23 19:58:43,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-23 19:58:43,796 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-23 19:58:43,797 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-23 19:58:43,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-23 19:58:43,813 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-23 19:58:43,814 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1089098131:90 , srv1130237638:90 , srv1174947418:90 , srv1316945850:90 , srv1336479784:90 , srv1369475543:90 , srv1424479901:90 , srv1479243937:90 , srv1509270064:90 , srv1521548240:90 , srv1642903489:90 , srv1656220104:90 , srv1703865817:90 , srv1802402542:90 , srv1816814887:90 , srv1871232203:90 , srv1994705263:90 , srv2002136700:90 , srv2033681713:90 , srv2090990781:90 , srv2116692923:90 , srv2120809637:90 , srv225372216:90 , srv363309005:90 , srv390744273:90 , srv453844452:90 , srv653220813:90 , srv662958393:90 , srv70996758:90 , srv913901453:90 }
2023-03-23 19:58:43,828 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=124 (was 279), ProcessCount=169 (was 169), AvailableMemoryMB=5821 (was 5931)