2023-11-24 04:53:19,228 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-24 04:53:19,545 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-24 04:53:19,665 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-24 04:53:19,732 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-24 04:53:19,740 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-24 04:53:19,758 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=171, AvailableMemoryMB=8076
2023-11-24 04:53:19,760 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-24 04:53:19,760 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-24 04:53:19,892 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1013584163:84 , srv104158190:84 , srv109622607:84 , srv1098818455:84 , srv1146143020:84 , srv1161390050:84 , srv1194469593:84 , srv1202305358:84 , srv1238022646:84 , srv1243743892:84 , srv126033958:84 , srv1272576316:84 , srv127578642:84 , srv1276905427:84 , srv127771688:84 , srv1286357792:84 , srv131617998:84 , srv1349433830:84 , srv1361270830:84 , srv1362216037:84 , srv1363020809:84 , srv1367082548:84 , srv1390074509:84 , srv1432334006:84 , srv1453321488:84 , srv1455383751:84 , srv159022092:84 , srv1598180856:84 , srv1619145031:84 , srv163387271:84 , srv1636932486:84 , srv1649522635:84 , srv1653921115:84 , srv1682150329:84 , srv1718723907:84 , srv1753349816:84 , srv1789411445:84 , srv1789438577:84 , srv1824430297:84 , srv1878751103:84 , srv1880264831:84 , srv1883168238:84 , srv189870667:84 , srv1905753741:84 , srv1921951904:84 , srv1963477402:84 , srv201173327:84 , srv2027658380:84 , srv2027852049:84 , srv2062364561:84 , srv2104559256:84 , srv2119129897:84 , srv2123744174:84 , srv222227978:84 , srv246401350:84 , srv252227117:84 , srv272412523:84 , srv306871868:84 , srv318841872:84 , srv323795980:84 , srv35174308:84 , srv366310689:84 , srv373275862:84 , srv374837839:84 , srv411943967:84 , srv411969215:84 , srv459689675:84 , srv465776310:84 , srv479915622:84 , srv527274387:84 , srv561589647:84 , srv571089081:84 , srv602327956:84 , srv608801250:84 , srv617608447:84 , srv667372761:84 , srv667863295:84 , srv708759165:84 , srv723491453:84 , srv723849594:84 , srv749312244:84 , srv754561318:84 , srv757139117:84 , srv778338021:84 , srv779301117:84 , srv795132884:84 , srv809019994:84 , srv829484448:84 , srv834146516:84 , srv856388136:84 , srv888156631:84 , srv890590962:84 , srv903931215:84 , srv904085328:84 , srv914976492:84 , srv922842392:84 , srv93874545:84 , srv971746237:84 , srv998976363:84 , srv1889898269:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-24 04:53:19,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:19,915 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1238022646=8, srv374837839=64, srv749312244=81, srv971746237=98, srv1453321488=24, srv1243743892=9, srv1455383751=25, srv323795980=60, srv1905753741=44, srv2062364561=50, srv2119129897=52, srv2123744174=53, srv1963477402=46, srv1653921115=32, srv479915622=69, srv834146516=89, srv104158190=1, srv222227978=54, srv252227117=56, srv2027852049=49, srv1349433830=17, srv829484448=88, srv1636932486=30, srv778338021=84, srv1682150329=33, srv779301117=85, srv1146143020=4, srv411969215=66, srv1432334006=23, srv1880264831=40, srv1363020809=20, srv723849594=80, srv2027658380=48, srv922842392=96, srv667372761=76, srv1789411445=36, srv890590962=92, srv608801250=74, srv1921951904=45, srv318841872=59, srv904085328=94, srv667863295=77, srv127771688=14, srv1390074509=22, srv1098818455=3, srv757139117=83, srv1598180856=27, srv1276905427=13, srv1789438577=37, srv708759165=78, srv1367082548=21, srv1824430297=38, srv246401350=55, srv93874545=97, srv109622607=2, srv1753349816=35, srv754561318=82, srv1272576316=11, srv1878751103=39, srv602327956=73, srv914976492=95, srv1161390050=5, srv373275862=63, srv809019994=87, srv465776310=68, srv126033958=10, srv1286357792=15, srv617608447=75, srv459689675=67, srv795132884=86, srv1883168238=41, srv127578642=12, srv723491453=79, srv1194469593=6, srv527274387=70, srv159022092=26, srv1619145031=28, srv411943967=65, srv1202305358=7, srv366310689=62, srv1718723907=34, srv131617998=16, srv201173327=47, srv561589647=71, srv1649522635=31, srv272412523=57, srv1013584163=0, srv2104559256=51, srv888156631=91, srv1362216037=19, srv163387271=29, srv306871868=58, srv856388136=90, srv571089081=72, srv1889898269=42, srv189870667=43, srv1361270830=18, srv35174308=61, srv998976363=99, srv903931215=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:19,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 04:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 04:53:19,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 04:53:19,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 04:53:20,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 04:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 04:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 04:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 04:53:20,200 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:20,201 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 04:53:20,202 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 04:53:20,623 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 591 ms to try 5000 different iterations.  Found a solution that moves 4693 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17631406450252937. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03770740127265468, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057894210577624124, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5214444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03770740127265468, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17133333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23983333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:20,649 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1238022646:82 , srv749312244:82 , srv829484448:82 , srv1146143020:83 , srv1161390050:83 , srv1718723907:83 , srv1789438577:83 , srv1883168238:83 , srv1963477402:83 , srv373275862:83 , srv617608447:83 , srv779301117:83 , srv890590962:83 , srv971746237:83 , srv1194469593:84 , srv1202305358:84 , srv1880264831:84 , srv2027852049:84 , srv323795980:84 , srv667863295:84 , srv127578642:85 , srv1362216037:85 , srv1363020809:85 , srv159022092:85 , srv1824430297:85 , srv189870667:85 , srv246401350:85 , srv252227117:85 , srv411943967:85 , srv411969215:85 , srv479915622:85 , srv757139117:85 , srv1013584163:86 , srv131617998:86 , srv1619145031:86 , srv163387271:86 , srv1905753741:86 , srv2104559256:86 , srv2119129897:86 , srv374837839:86 , srv527274387:86 , srv561589647:86 , srv608801250:86 , srv667372761:86 , srv708759165:86 , srv723491453:86 , srv795132884:86 , srv809019994:86 , srv922842392:86 , srv93874545:86 , srv998976363:86 , srv1243743892:87 , srv126033958:87 , srv1276905427:87 , srv127771688:87 , srv1453321488:87 , srv1455383751:87 , srv1653921115:87 , srv2062364561:87 , srv318841872:87 , srv778338021:87 , srv856388136:87 , srv888156631:87 , srv914976492:87 , srv1098818455:88 , srv1349433830:88 , srv1367082548:88 , srv1390074509:88 , srv1432334006:88 , srv1598180856:88 , srv1636932486:88 , srv1649522635:88 , srv1789411445:88 , srv2123744174:88 , srv459689675:88 , srv904085328:88 , srv109622607:89 , srv1272576316:89 , srv1361270830:89 , srv1682150329:89 , srv1753349816:89 , srv1878751103:89 , srv1921951904:89 , srv201173327:89 , srv306871868:89 , srv35174308:89 , srv366310689:89 , srv571089081:89 , srv602327956:89 , srv834146516:89 , srv2027658380:90 , srv465776310:90 , srv723849594:90 , srv903931215:90 , srv1286357792:91 , srv754561318:91 , srv272412523:93 , srv104158190:94 , srv222227978:96 , srv1889898269:425 }
2023-11-24 04:53:20,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:20,655 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1238022646=8, srv374837839=64, srv749312244=81, srv971746237=98, srv1453321488=24, srv1243743892=9, srv1455383751=25, srv323795980=60, srv1905753741=44, srv2062364561=50, srv2119129897=52, srv2123744174=53, srv1963477402=46, srv1653921115=32, srv479915622=69, srv834146516=89, srv104158190=1, srv222227978=54, srv252227117=56, srv2027852049=49, srv1349433830=17, srv829484448=88, srv1636932486=30, srv778338021=84, srv1682150329=33, srv779301117=85, srv1146143020=4, srv411969215=66, srv1432334006=23, srv1880264831=40, srv1363020809=20, srv723849594=80, srv2027658380=48, srv922842392=96, srv667372761=76, srv1789411445=36, srv890590962=92, srv608801250=74, srv1921951904=45, srv318841872=59, srv904085328=94, srv667863295=77, srv127771688=14, srv1390074509=22, srv1098818455=3, srv757139117=83, srv1598180856=27, srv1276905427=13, srv1789438577=37, srv708759165=78, srv1367082548=21, srv1824430297=38, srv246401350=55, srv93874545=97, srv109622607=2, srv1753349816=35, srv754561318=82, srv1272576316=11, srv1878751103=39, srv602327956=73, srv914976492=95, srv1161390050=5, srv373275862=63, srv809019994=87, srv465776310=68, srv126033958=10, srv1286357792=15, srv617608447=75, srv459689675=67, srv795132884=86, srv1883168238=41, srv127578642=12, srv723491453=79, srv1194469593=6, srv527274387=70, srv159022092=26, srv1619145031=28, srv411943967=65, srv1202305358=7, srv366310689=62, srv1718723907=34, srv131617998=16, srv201173327=47, srv561589647=71, srv1649522635=31, srv272412523=57, srv1013584163=0, srv2104559256=51, srv888156631=91, srv1362216037=19, srv163387271=29, srv306871868=58, srv856388136=90, srv571089081=72, srv1889898269=42, srv189870667=43, srv1361270830=18, srv35174308=61, srv998976363=99, srv903931215=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:20,679 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:20,680 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:20,681 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:20,682 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 04:53:20,683 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 04:53:20,684 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 04:53:20,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 04:53:20,686 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 04:53:20,687 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 04:53:20,688 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 04:53:20,689 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 04:53:20,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 04:53:20,690 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 04:53:20,859 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03770740127265468, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057894210577624124, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03770740127265468, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17133333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23983333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:20,859 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 04:53:20,859 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17628120341417527, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03770740127265468, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057894210577624124, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03770740127265468, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17133333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23983333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 04:53:21,154 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 432 ms to try 5000 different iterations.  Found a solution that moves 2625 regions; Going from a computed imbalance of 0.17628120341417527 to a new imbalance of 0.013931788476844054. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022731537198118608); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039929792482827416, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2916666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022731537198118608); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:21,171 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv222227978:74 , srv754561318:79 , srv1286357792:80 , srv723849594:80 , srv104158190:81 , srv1753349816:81 , srv903931215:81 , srv1878751103:82 , srv35174308:82 , srv465776310:82 , srv834146516:82 , srv109622607:83 , srv1361270830:83 , srv1390074509:83 , srv1455383751:83 , srv1598180856:83 , srv1636932486:83 , srv1649522635:83 , srv1921951904:83 , srv272412523:83 , srv306871868:83 , srv459689675:83 , srv602327956:83 , srv904085328:83 , srv1098818455:84 , srv1272576316:84 , srv131617998:84 , srv1453321488:84 , srv1653921115:84 , srv1905753741:84 , srv2027658380:84 , srv318841872:84 , srv366310689:84 , srv888156631:84 , srv914976492:84 , srv998976363:84 , srv1243743892:85 , srv1276905427:85 , srv1349433830:85 , srv1363020809:85 , srv1367082548:85 , srv1432334006:85 , srv163387271:85 , srv1789411445:85 , srv189870667:85 , srv201173327:85 , srv2062364561:85 , srv2119129897:85 , srv2123744174:85 , srv411943967:85 , srv527274387:85 , srv561589647:85 , srv757139117:85 , srv856388136:85 , srv93874545:85 , srv126033958:86 , srv1362216037:86 , srv159022092:86 , srv1824430297:86 , srv2104559256:86 , srv374837839:86 , srv608801250:86 , srv667372761:86 , srv708759165:86 , srv795132884:86 , srv1619145031:87 , srv1682150329:87 , srv1880264831:87 , srv323795980:87 , srv373275862:87 , srv411969215:87 , srv479915622:87 , srv571089081:87 , srv667863295:87 , srv723491453:87 , srv778338021:87 , srv890590962:87 , srv922842392:87 , srv1146143020:88 , srv1161390050:88 , srv1202305358:88 , srv127771688:88 , srv1789438577:88 , srv1883168238:88 , srv2027852049:88 , srv252227117:88 , srv1013584163:89 , srv1194469593:89 , srv127578642:89 , srv1963477402:89 , srv246401350:89 , srv809019994:89 , srv1238022646:90 , srv1718723907:90 , srv617608447:90 , srv749312244:90 , srv779301117:90 , srv829484448:92 , srv971746237:92 , srv1889898269:551 }
2023-11-24 04:53:21,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:21,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1238022646=8, srv374837839=64, srv749312244=81, srv971746237=98, srv1453321488=24, srv1243743892=9, srv1455383751=25, srv323795980=60, srv1905753741=44, srv2062364561=50, srv2119129897=52, srv2123744174=53, srv1963477402=46, srv1653921115=32, srv479915622=69, srv834146516=89, srv104158190=1, srv222227978=54, srv252227117=56, srv2027852049=49, srv1349433830=17, srv829484448=88, srv1636932486=30, srv778338021=84, srv1682150329=33, srv779301117=85, srv1146143020=4, srv411969215=66, srv1432334006=23, srv1880264831=40, srv1363020809=20, srv723849594=80, srv2027658380=48, srv922842392=96, srv667372761=76, srv1789411445=36, srv890590962=92, srv608801250=74, srv1921951904=45, srv318841872=59, srv904085328=94, srv667863295=77, srv127771688=14, srv1390074509=22, srv1098818455=3, srv757139117=83, srv1598180856=27, srv1276905427=13, srv1789438577=37, srv708759165=78, srv1367082548=21, srv1824430297=38, srv246401350=55, srv93874545=97, srv109622607=2, srv1753349816=35, srv754561318=82, srv1272576316=11, srv1878751103=39, srv602327956=73, srv914976492=95, srv1161390050=5, srv373275862=63, srv809019994=87, srv465776310=68, srv126033958=10, srv1286357792=15, srv617608447=75, srv459689675=67, srv795132884=86, srv1883168238=41, srv127578642=12, srv723491453=79, srv1194469593=6, srv527274387=70, srv159022092=26, srv1619145031=28, srv411943967=65, srv1202305358=7, srv366310689=62, srv1718723907=34, srv131617998=16, srv201173327=47, srv561589647=71, srv1649522635=31, srv272412523=57, srv1013584163=0, srv2104559256=51, srv888156631=91, srv1362216037=19, srv163387271=29, srv306871868=58, srv856388136=90, srv571089081=72, srv1889898269=42, srv189870667=43, srv1361270830=18, srv35174308=61, srv998976363=99, srv903931215=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:21,198 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:21,199 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:21,200 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:21,201 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 04:53:21,202 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 04:53:21,203 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 04:53:21,204 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 04:53:21,205 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 04:53:21,206 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 04:53:21,207 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 04:53:21,208 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 04:53:21,209 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 04:53:21,367 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022731537198118608); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039929792482827416, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022731537198118608); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:21,367 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 04:53:21,367 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013913407833986698, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022731537198118608); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.039929792482827416, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022731537198118608); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01075); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.043833333333333335, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 04:53:21,789 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 553 ms to try 5000 different iterations.  Found a solution that moves 969 regions; Going from a computed imbalance of 0.013913407833986698 to a new imbalance of 5.223970876603744E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006207534371868385); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00956987416753698); MoveCostFunction : (multiplier=7.0, imbalance=0.10766666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006207534371868385); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:21,792 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1682150329:81 , srv571089081:81 , srv127771688:82 , srv104158190:83 , srv1272576316:83 , srv246401350:83 , srv272412523:83 , srv809019994:83 , srv829484448:83 , srv971746237:83 , srv1013584163:84 , srv1194469593:84 , srv126033958:84 , srv127578642:84 , srv1432334006:84 , srv1619145031:84 , srv1789411445:84 , srv201173327:84 , srv2027658380:84 , srv366310689:84 , srv617608447:84 , srv778338021:84 , srv1098818455:85 , srv1202305358:85 , srv1243743892:85 , srv1276905427:85 , srv1349433830:85 , srv1361270830:85 , srv1367082548:85 , srv1718723907:85 , srv1824430297:85 , srv2027852049:85 , srv2062364561:85 , srv2123744174:85 , srv252227117:85 , srv306871868:85 , srv374837839:85 , srv465776310:85 , srv479915622:85 , srv608801250:85 , srv708759165:85 , srv723491453:85 , srv779301117:85 , srv795132884:85 , srv856388136:85 , srv922842392:85 , srv109622607:86 , srv1146143020:86 , srv1238022646:86 , srv1286357792:86 , srv1362216037:86 , srv1390074509:86 , srv1453321488:86 , srv159022092:86 , srv1598180856:86 , srv163387271:86 , srv1636932486:86 , srv1653921115:86 , srv1789438577:86 , srv1921951904:86 , srv1963477402:86 , srv2104559256:86 , srv35174308:86 , srv411969215:86 , srv459689675:86 , srv527274387:86 , srv561589647:86 , srv602327956:86 , srv667372761:86 , srv667863295:86 , srv749312244:86 , srv757139117:86 , srv93874545:86 , srv1161390050:87 , srv1363020809:87 , srv1455383751:87 , srv1649522635:87 , srv1878751103:87 , srv1880264831:87 , srv1883168238:87 , srv189870667:87 , srv1905753741:87 , srv2119129897:87 , srv318841872:87 , srv323795980:87 , srv373275862:87 , srv834146516:87 , srv888156631:87 , srv890590962:87 , srv903931215:87 , srv904085328:87 , srv914976492:87 , srv998976363:87 , srv131617998:88 , srv1753349816:88 , srv222227978:88 , srv411943967:88 , srv723849594:88 , srv754561318:88 , srv1889898269:537 }
2023-11-24 04:53:21,798 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:21,799 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1238022646=8, srv374837839=64, srv749312244=81, srv971746237=98, srv1453321488=24, srv1243743892=9, srv1455383751=25, srv323795980=60, srv1905753741=44, srv2062364561=50, srv2119129897=52, srv2123744174=53, srv1963477402=46, srv1653921115=32, srv479915622=69, srv834146516=89, srv104158190=1, srv222227978=54, srv252227117=56, srv2027852049=49, srv1349433830=17, srv829484448=88, srv1636932486=30, srv778338021=84, srv1682150329=33, srv779301117=85, srv1146143020=4, srv411969215=66, srv1432334006=23, srv1880264831=40, srv1363020809=20, srv723849594=80, srv2027658380=48, srv922842392=96, srv667372761=76, srv1789411445=36, srv890590962=92, srv608801250=74, srv1921951904=45, srv318841872=59, srv904085328=94, srv667863295=77, srv127771688=14, srv1390074509=22, srv1098818455=3, srv757139117=83, srv1598180856=27, srv1276905427=13, srv1789438577=37, srv708759165=78, srv1367082548=21, srv1824430297=38, srv246401350=55, srv93874545=97, srv109622607=2, srv1753349816=35, srv754561318=82, srv1272576316=11, srv1878751103=39, srv602327956=73, srv914976492=95, srv1161390050=5, srv373275862=63, srv809019994=87, srv465776310=68, srv126033958=10, srv1286357792=15, srv617608447=75, srv459689675=67, srv795132884=86, srv1883168238=41, srv127578642=12, srv723491453=79, srv1194469593=6, srv527274387=70, srv159022092=26, srv1619145031=28, srv411943967=65, srv1202305358=7, srv366310689=62, srv1718723907=34, srv131617998=16, srv201173327=47, srv561589647=71, srv1649522635=31, srv272412523=57, srv1013584163=0, srv2104559256=51, srv888156631=91, srv1362216037=19, srv163387271=29, srv306871868=58, srv856388136=90, srv571089081=72, srv1889898269=42, srv189870667=43, srv1361270830=18, srv35174308=61, srv998976363=99, srv903931215=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:21,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:21,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:21,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:21,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:21,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:21,818 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:21,819 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:21,820 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:21,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 04:53:21,822 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 04:53:21,823 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 04:53:21,824 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 04:53:21,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 04:53:21,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 04:53:21,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 04:53:21,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 04:53:21,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 04:53:21,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 04:53:21,988 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006207534371868385); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00956987416753698); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006207534371868385); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:21,988 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 04:53:21,989 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.156120046398871E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006207534371868385); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.00956987416753698); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006207534371868385); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.6666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 04:53:22,391 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 546 ms to try 5000 different iterations.  Found a solution that moves 252 regions; Going from a computed imbalance of 5.156120046398871E-4 to a new imbalance of 4.458850541359721E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.028, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:22,392 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv222227978:83 , srv272412523:83 , srv1013584163:84 , srv104158190:84 , srv109622607:84 , srv1161390050:84 , srv1202305358:84 , srv1238022646:84 , srv126033958:84 , srv127578642:84 , srv127771688:84 , srv131617998:84 , srv1349433830:84 , srv1367082548:84 , srv1649522635:84 , srv1718723907:84 , srv1753349816:84 , srv1789438577:84 , srv1878751103:84 , srv1880264831:84 , srv1921951904:84 , srv1963477402:84 , srv201173327:84 , srv2027658380:84 , srv2104559256:84 , srv2119129897:84 , srv2123744174:84 , srv252227117:84 , srv306871868:84 , srv323795980:84 , srv411943967:84 , srv602327956:84 , srv667372761:84 , srv723491453:84 , srv749312244:84 , srv754561318:84 , srv778338021:84 , srv779301117:84 , srv834146516:84 , srv888156631:84 , srv903931215:84 , srv904085328:84 , srv914976492:84 , srv922842392:84 , srv971746237:84 , srv1098818455:85 , srv1146143020:85 , srv1194469593:85 , srv1243743892:85 , srv1272576316:85 , srv1276905427:85 , srv1286357792:85 , srv1361270830:85 , srv1362216037:85 , srv1363020809:85 , srv1390074509:85 , srv1432334006:85 , srv1453321488:85 , srv1455383751:85 , srv159022092:85 , srv1598180856:85 , srv1619145031:85 , srv163387271:85 , srv1636932486:85 , srv1653921115:85 , srv1682150329:85 , srv1824430297:85 , srv1883168238:85 , srv189870667:85 , srv1905753741:85 , srv2027852049:85 , srv2062364561:85 , srv246401350:85 , srv318841872:85 , srv35174308:85 , srv366310689:85 , srv373275862:85 , srv374837839:85 , srv411969215:85 , srv459689675:85 , srv465776310:85 , srv479915622:85 , srv527274387:85 , srv561589647:85 , srv571089081:85 , srv608801250:85 , srv617608447:85 , srv667863295:85 , srv708759165:85 , srv723849594:85 , srv795132884:85 , srv809019994:85 , srv829484448:85 , srv890590962:85 , srv93874545:85 , srv998976363:85 , srv1789411445:86 , srv757139117:86 , srv856388136:86 , srv1889898269:629 }
2023-11-24 04:53:22,397 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:22,398 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1238022646=8, srv374837839=64, srv749312244=81, srv971746237=98, srv1453321488=24, srv1243743892=9, srv1455383751=25, srv323795980=60, srv1905753741=44, srv2062364561=50, srv2119129897=52, srv2123744174=53, srv1963477402=46, srv1653921115=32, srv479915622=69, srv834146516=89, srv104158190=1, srv222227978=54, srv252227117=56, srv2027852049=49, srv1349433830=17, srv829484448=88, srv1636932486=30, srv778338021=84, srv1682150329=33, srv779301117=85, srv1146143020=4, srv411969215=66, srv1432334006=23, srv1880264831=40, srv1363020809=20, srv723849594=80, srv2027658380=48, srv922842392=96, srv667372761=76, srv1789411445=36, srv890590962=92, srv608801250=74, srv1921951904=45, srv318841872=59, srv904085328=94, srv667863295=77, srv127771688=14, srv1390074509=22, srv1098818455=3, srv757139117=83, srv1598180856=27, srv1276905427=13, srv1789438577=37, srv708759165=78, srv1367082548=21, srv1824430297=38, srv246401350=55, srv93874545=97, srv109622607=2, srv1753349816=35, srv754561318=82, srv1272576316=11, srv1878751103=39, srv602327956=73, srv914976492=95, srv1161390050=5, srv373275862=63, srv809019994=87, srv465776310=68, srv126033958=10, srv1286357792=15, srv617608447=75, srv459689675=67, srv795132884=86, srv1883168238=41, srv127578642=12, srv723491453=79, srv1194469593=6, srv527274387=70, srv159022092=26, srv1619145031=28, srv411943967=65, srv1202305358=7, srv366310689=62, srv1718723907=34, srv131617998=16, srv201173327=47, srv561589647=71, srv1649522635=31, srv272412523=57, srv1013584163=0, srv2104559256=51, srv888156631=91, srv1362216037=19, srv163387271=29, srv306871868=58, srv856388136=90, srv571089081=72, srv1889898269=42, srv189870667=43, srv1361270830=18, srv35174308=61, srv998976363=99, srv903931215=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:22,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:22,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:22,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:22,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:22,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:22,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:22,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:22,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:22,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:22,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:22,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:22,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:22,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:22,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 04:53:22,421 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 04:53:22,422 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 04:53:22,423 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 04:53:22,424 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 04:53:22,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 04:53:22,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 04:53:22,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 04:53:22,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 04:53:22,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 04:53:22,425 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 04:53:22,426 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 04:53:22,427 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 04:53:22,540 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:22,540 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-24 04:53:22,541 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=4.2823963699290913E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0023688968483956715); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=3.333333333333333E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-24 04:53:22,862 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 426 ms to try 5000 different iterations.  Found a solution that moves 40 regions; Going from a computed imbalance of 4.2823963699290913E-5 to a new imbalance of 9.07327622571662E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0044444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:22,863 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1013584163:83 , srv1789411445:83 , srv1883168238:83 , srv318841872:83 , srv411969215:83 , srv723849594:83 , srv809019994:83 , srv856388136:83 , srv104158190:84 , srv109622607:84 , srv1098818455:84 , srv1146143020:84 , srv1161390050:84 , srv1194469593:84 , srv1238022646:84 , srv1243743892:84 , srv127578642:84 , srv1276905427:84 , srv1286357792:84 , srv131617998:84 , srv1349433830:84 , srv1361270830:84 , srv1362216037:84 , srv1363020809:84 , srv1367082548:84 , srv1390074509:84 , srv1432334006:84 , srv1453321488:84 , srv1455383751:84 , srv159022092:84 , srv1598180856:84 , srv1619145031:84 , srv163387271:84 , srv1636932486:84 , srv1649522635:84 , srv1653921115:84 , srv1682150329:84 , srv1718723907:84 , srv1753349816:84 , srv1878751103:84 , srv1880264831:84 , srv189870667:84 , srv1905753741:84 , srv1921951904:84 , srv1963477402:84 , srv201173327:84 , srv2027658380:84 , srv2027852049:84 , srv2062364561:84 , srv2104559256:84 , srv2119129897:84 , srv2123744174:84 , srv246401350:84 , srv252227117:84 , srv272412523:84 , srv323795980:84 , srv35174308:84 , srv366310689:84 , srv373275862:84 , srv374837839:84 , srv411943967:84 , srv459689675:84 , srv465776310:84 , srv479915622:84 , srv527274387:84 , srv561589647:84 , srv571089081:84 , srv602327956:84 , srv608801250:84 , srv617608447:84 , srv667372761:84 , srv667863295:84 , srv708759165:84 , srv723491453:84 , srv749312244:84 , srv754561318:84 , srv757139117:84 , srv778338021:84 , srv779301117:84 , srv795132884:84 , srv829484448:84 , srv834146516:84 , srv888156631:84 , srv890590962:84 , srv903931215:84 , srv904085328:84 , srv914976492:84 , srv922842392:84 , srv93874545:84 , srv971746237:84 , srv998976363:84 , srv1202305358:85 , srv126033958:85 , srv1272576316:85 , srv127771688:85 , srv1789438577:85 , srv1824430297:85 , srv222227978:85 , srv306871868:85 , srv1889898269:684 }
2023-11-24 04:53:22,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:22,867 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1238022646=8, srv374837839=64, srv749312244=81, srv971746237=98, srv1453321488=24, srv1243743892=9, srv1455383751=25, srv323795980=60, srv1905753741=44, srv2062364561=50, srv2119129897=52, srv2123744174=53, srv1963477402=46, srv1653921115=32, srv479915622=69, srv834146516=89, srv104158190=1, srv222227978=54, srv252227117=56, srv2027852049=49, srv1349433830=17, srv829484448=88, srv1636932486=30, srv778338021=84, srv1682150329=33, srv779301117=85, srv1146143020=4, srv411969215=66, srv1432334006=23, srv1880264831=40, srv1363020809=20, srv723849594=80, srv2027658380=48, srv922842392=96, srv667372761=76, srv1789411445=36, srv890590962=92, srv608801250=74, srv1921951904=45, srv318841872=59, srv904085328=94, srv667863295=77, srv127771688=14, srv1390074509=22, srv1098818455=3, srv757139117=83, srv1598180856=27, srv1276905427=13, srv1789438577=37, srv708759165=78, srv1367082548=21, srv1824430297=38, srv246401350=55, srv93874545=97, srv109622607=2, srv1753349816=35, srv754561318=82, srv1272576316=11, srv1878751103=39, srv602327956=73, srv914976492=95, srv1161390050=5, srv373275862=63, srv809019994=87, srv465776310=68, srv126033958=10, srv1286357792=15, srv617608447=75, srv459689675=67, srv795132884=86, srv1883168238=41, srv127578642=12, srv723491453=79, srv1194469593=6, srv527274387=70, srv159022092=26, srv1619145031=28, srv411943967=65, srv1202305358=7, srv366310689=62, srv1718723907=34, srv131617998=16, srv201173327=47, srv561589647=71, srv1649522635=31, srv272412523=57, srv1013584163=0, srv2104559256=51, srv888156631=91, srv1362216037=19, srv163387271=29, srv306871868=58, srv856388136=90, srv571089081=72, srv1889898269=42, srv189870667=43, srv1361270830=18, srv35174308=61, srv998976363=99, srv903931215=93} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:22,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:22,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:22,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:22,893 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-24 04:53:22,894 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-24 04:53:22,895 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-24 04:53:22,896 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-24 04:53:22,897 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-24 04:53:22,898 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-24 04:53:22,899 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-24 04:53:22,900 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-24 04:53:23,026 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=8.793190239318795E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:23,026 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1013584163:83 , srv1789411445:83 , srv1883168238:83 , srv318841872:83 , srv411969215:83 , srv723849594:83 , srv809019994:83 , srv856388136:83 , srv104158190:84 , srv109622607:84 , srv1098818455:84 , srv1146143020:84 , srv1161390050:84 , srv1194469593:84 , srv1238022646:84 , srv1243743892:84 , srv127578642:84 , srv1276905427:84 , srv1286357792:84 , srv131617998:84 , srv1349433830:84 , srv1361270830:84 , srv1362216037:84 , srv1363020809:84 , srv1367082548:84 , srv1390074509:84 , srv1432334006:84 , srv1453321488:84 , srv1455383751:84 , srv159022092:84 , srv1598180856:84 , srv1619145031:84 , srv163387271:84 , srv1636932486:84 , srv1649522635:84 , srv1653921115:84 , srv1682150329:84 , srv1718723907:84 , srv1753349816:84 , srv1878751103:84 , srv1880264831:84 , srv189870667:84 , srv1905753741:84 , srv1921951904:84 , srv1963477402:84 , srv201173327:84 , srv2027658380:84 , srv2027852049:84 , srv2062364561:84 , srv2104559256:84 , srv2119129897:84 , srv2123744174:84 , srv246401350:84 , srv252227117:84 , srv272412523:84 , srv323795980:84 , srv35174308:84 , srv366310689:84 , srv373275862:84 , srv374837839:84 , srv411943967:84 , srv459689675:84 , srv465776310:84 , srv479915622:84 , srv527274387:84 , srv561589647:84 , srv571089081:84 , srv602327956:84 , srv608801250:84 , srv617608447:84 , srv667372761:84 , srv667863295:84 , srv708759165:84 , srv723491453:84 , srv749312244:84 , srv754561318:84 , srv757139117:84 , srv778338021:84 , srv779301117:84 , srv795132884:84 , srv829484448:84 , srv834146516:84 , srv888156631:84 , srv890590962:84 , srv903931215:84 , srv904085328:84 , srv914976492:84 , srv922842392:84 , srv93874545:84 , srv971746237:84 , srv998976363:84 , srv1202305358:85 , srv126033958:85 , srv1272576316:85 , srv127771688:85 , srv1789438577:85 , srv1824430297:85 , srv222227978:85 , srv306871868:85 , srv1889898269:684 }
2023-11-24 04:53:23,055 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=360 (was 339) - SystemLoadAverage LEAK? -, ProcessCount=171 (was 171), AvailableMemoryMB=7090 (was 8076)
2023-11-24 04:53:23,062 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=360, ProcessCount=171, AvailableMemoryMB=7090
2023-11-24 04:53:23,063 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-24 04:53:23,064 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-24 04:53:23,077 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1042615590:84 , srv1059578990:84 , srv1264965879:84 , srv1312790503:84 , srv1409393055:84 , srv1417210763:84 , srv1447219842:84 , srv1641483000:84 , srv1724839698:84 , srv176107937:84 , srv1819012600:84 , srv1821144230:84 , srv1924758362:84 , srv1989722881:84 , srv2039999464:84 , srv207501570:84 , srv335281788:84 , srv386614782:84 , srv439057432:84 , srv448890617:84 , srv476727599:84 , srv511283445:84 , srv667028991:84 , srv741128455:84 , srv878897687:84 , srv881278112:84 , srv932290103:84 , srv941366592:84 , srv985687324:84 , srv1869036819:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-24 04:53:23,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:53:23,079 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1417210763=5, srv207501570=16, srv985687324=29, srv448890617=20, srv881278112=26, srv1724839698=8, srv1059578990=1, srv335281788=17, srv1409393055=4, srv1447219842=6, srv667028991=23, srv932290103=27, srv1312790503=3, srv1821144230=11, srv476727599=21, srv1264965879=2, srv1989722881=14, srv878897687=25, srv741128455=24, srv2039999464=15, srv1042615590=0, srv1641483000=7, srv1869036819=12, srv511283445=22, srv1819012600=10, srv439057432=19, srv386614782=18, srv941366592=28, srv176107937=9, srv1924758362=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:53:23,095 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:53:23,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:53:23,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:53:23,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:53:23,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:53:23,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:53:23,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-24 04:53:23,123 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1448322745783917, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:53:23,124 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912649405032268, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1448322745783917, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-24 04:55:23,102 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5738428 different iterations.  Found a solution that moves 1854 regions; Going from a computed imbalance of 0.9912649405032268 to a new imbalance of 4.3273284898463826E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6866666666666666, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:55:23,110 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1042615590:90 , srv1059578990:90 , srv1264965879:90 , srv1312790503:90 , srv1409393055:90 , srv1417210763:90 , srv1447219842:90 , srv1641483000:90 , srv1724839698:90 , srv176107937:90 , srv1819012600:90 , srv1821144230:90 , srv1869036819:90 , srv1924758362:90 , srv1989722881:90 , srv2039999464:90 , srv207501570:90 , srv335281788:90 , srv386614782:90 , srv439057432:90 , srv448890617:90 , srv476727599:90 , srv511283445:90 , srv667028991:90 , srv741128455:90 , srv878897687:90 , srv881278112:90 , srv932290103:90 , srv941366592:90 , srv985687324:90 }
2023-11-24 04:55:23,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-24 04:55:23,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1417210763=5, srv207501570=16, srv985687324=29, srv448890617=20, srv881278112=26, srv1724839698=8, srv1059578990=1, srv335281788=17, srv1409393055=4, srv1447219842=6, srv667028991=23, srv932290103=27, srv1312790503=3, srv1821144230=11, srv476727599=21, srv1264965879=2, srv1989722881=14, srv878897687=25, srv741128455=24, srv2039999464=15, srv1042615590=0, srv1641483000=7, srv1869036819=12, srv511283445=22, srv1819012600=10, srv439057432=19, srv386614782=18, srv941366592=28, srv176107937=9, srv1924758362=13} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-24 04:55:23,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-24 04:55:23,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-24 04:55:23,131 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-24 04:55:23,131 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1042615590:90 , srv1059578990:90 , srv1264965879:90 , srv1312790503:90 , srv1409393055:90 , srv1417210763:90 , srv1447219842:90 , srv1641483000:90 , srv1724839698:90 , srv176107937:90 , srv1819012600:90 , srv1821144230:90 , srv1869036819:90 , srv1924758362:90 , srv1989722881:90 , srv2039999464:90 , srv207501570:90 , srv335281788:90 , srv386614782:90 , srv439057432:90 , srv448890617:90 , srv476727599:90 , srv511283445:90 , srv667028991:90 , srv741128455:90 , srv878897687:90 , srv881278112:90 , srv932290103:90 , srv941366592:90 , srv985687324:90 }
2023-11-24 04:55:23,147 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 360), ProcessCount=171 (was 171), AvailableMemoryMB=6503 (was 7090)