2023-11-19 02:55:15,138 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-19 02:55:15,409 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-19 02:55:15,524 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-19 02:55:15,593 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-19 02:55:15,600 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-19 02:55:15,613 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=177, AvailableMemoryMB=18653
2023-11-19 02:55:15,615 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-19 02:55:15,615 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-19 02:55:15,751 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1002566505:84 , srv103618542:84 , srv1058252969:84 , srv1090561271:84 , srv1093315112:84 , srv1112081858:84 , srv1149749254:84 , srv1152931340:84 , srv1159953757:84 , srv1228556515:84 , srv1277744863:84 , srv129915857:84 , srv1300499582:84 , srv1314530338:84 , srv1316331588:84 , srv1316951789:84 , srv132918371:84 , srv132979080:84 , srv1421991302:84 , srv1430938397:84 , srv1449107527:84 , srv1465160612:84 , srv147307745:84 , srv1514400855:84 , srv1533617238:84 , srv1599450477:84 , srv1603810695:84 , srv1610174961:84 , srv1618946733:84 , srv1646183935:84 , srv1663924588:84 , srv1671920660:84 , srv17515464:84 , srv1756032925:84 , srv1775063387:84 , srv1835066863:84 , srv1838832631:84 , srv1840926564:84 , srv1857176843:84 , srv1866229360:84 , srv1869482791:84 , srv193755933:84 , srv1946606357:84 , srv1969594994:84 , srv197626510:84 , srv1986495906:84 , srv1989369085:84 , srv2028042582:84 , srv202983218:84 , srv2036122239:84 , srv2036188759:84 , srv2089279086:84 , srv210225082:84 , srv2110262852:84 , srv2114294088:84 , srv2125043942:84 , srv2133464386:84 , srv225649486:84 , srv257589583:84 , srv258569830:84 , srv296599171:84 , srv296713995:84 , srv324767157:84 , srv334097614:84 , srv349886296:84 , srv357826191:84 , srv397492349:84 , srv4128677:84 , srv41557014:84 , srv432457306:84 , srv45853174:84 , srv465688639:84 , srv479775982:84 , srv481833907:84 , srv484756696:84 , srv504133567:84 , srv519393859:84 , srv547051573:84 , srv573480192:84 , srv588507424:84 , srv606293107:84 , srv617192622:84 , srv622975780:84 , srv633128340:84 , srv665282157:84 , srv718722465:84 , srv724533251:84 , srv727326198:84 , srv790234028:84 , srv795773821:84 , srv800014714:84 , srv815621785:84 , srv819295552:84 , srv82916138:84 , srv839501274:84 , srv839683146:84 , srv854311398:84 , srv878017628:84 , srv979084382:84 , srv673572888:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-19 02:55:15,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:55:15,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1300499582=12, srv1002566505=0, srv45853174=70, srv800014714=91, srv258569830=59, srv1465160612=21, srv1775063387=34, srv147307745=22, srv1093315112=4, srv1866229360=39, srv606293107=80, srv132918371=16, srv727326198=88, srv1090561271=3, srv17515464=32, srv4128677=67, srv839501274=95, srv1152931340=7, srv617192622=81, srv547051573=77, srv1618946733=28, srv41557014=68, srv819295552=93, srv296599171=60, srv979084382=99, srv1533617238=24, srv854311398=97, srv1838832631=36, srv519393859=76, srv210225082=52, srv2133464386=56, srv1058252969=2, srv2089279086=51, srv129915857=11, srv1857176843=38, srv1840926564=37, srv296713995=61, srv1112081858=5, srv349886296=64, srv1663924588=30, srv334097614=63, srv588507424=79, srv1603810695=26, srv132979080=17, srv795773821=90, srv1228556515=9, srv202983218=48, srv2028042582=47, srv1610174961=27, srv1316951789=15, srv2110262852=53, srv573480192=78, srv481833907=73, srv1514400855=23, srv665282157=84, srv1869482791=40, srv1989369085=46, srv815621785=92, srv484756696=74, srv2114294088=54, srv1277744863=10, srv197626510=44, srv465688639=71, srv1756032925=33, srv225649486=57, srv82916138=94, srv1149749254=6, srv1159953757=8, srv1835066863=35, srv2036122239=49, srv2125043942=55, srv1646183935=29, srv718722465=86, srv790234028=89, srv839683146=96, srv878017628=98, srv1314530338=13, srv504133567=75, srv1449107527=20, srv622975780=82, srv357826191=65, srv1599450477=25, srv673572888=85, srv1430938397=19, srv193755933=41, srv1671920660=31, srv1946606357=42, srv257589583=58, srv432457306=69, srv2036188759=50, srv1316331588=14, srv724533251=87, srv103618542=1, srv324767157=62, srv397492349=66, srv633128340=83, srv1969594994=43, srv1986495906=45, srv1421991302=18, srv479775982=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:55:15,844 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:55:15,845 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:55:15,846 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 02:55:15,847 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 02:55:15,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 02:55:15,849 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 02:55:15,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 02:55:15,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 02:55:15,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 02:55:15,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 02:55:15,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 02:55:15,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 02:55:15,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 02:55:15,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 02:55:15,854 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 02:55:16,101 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:16,101 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 02:55:16,102 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 02:55:16,557 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 682 ms to try 5000 different iterations.  Found a solution that moves 4663 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1791797483681744. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04150462317448463, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05550098329782828, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5181111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04150462317448463, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:16,568 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1112081858:82 , srv1838832631:82 , srv1002566505:83 , srv129915857:83 , srv17515464:83 , srv1986495906:83 , srv2028042582:83 , srv2036188759:83 , srv210225082:83 , srv257589583:83 , srv45853174:83 , srv727326198:83 , srv1149749254:84 , srv1159953757:84 , srv1277744863:84 , srv1599450477:84 , srv1671920660:84 , srv296599171:84 , srv334097614:84 , srv484756696:84 , srv504133567:84 , srv519393859:84 , srv606293107:84 , srv815621785:84 , srv839683146:84 , srv1090561271:85 , srv1300499582:85 , srv1316331588:85 , srv1316951789:85 , srv132918371:85 , srv1514400855:85 , srv1775063387:85 , srv1866229360:85 , srv1869482791:85 , srv2110262852:85 , srv2114294088:85 , srv258569830:85 , srv324767157:85 , srv465688639:85 , srv588507424:85 , srv718722465:85 , srv724533251:85 , srv800014714:85 , srv979084382:85 , srv103618542:86 , srv1058252969:86 , srv1093315112:86 , srv1314530338:86 , srv1533617238:86 , srv1618946733:86 , srv1857176843:86 , srv193755933:86 , srv197626510:86 , srv2089279086:86 , srv2133464386:86 , srv573480192:86 , srv665282157:86 , srv1152931340:87 , srv1430938397:87 , srv1465160612:87 , srv1603810695:87 , srv1646183935:87 , srv1663924588:87 , srv1946606357:87 , srv1969594994:87 , srv1989369085:87 , srv397492349:87 , srv479775982:87 , srv481833907:87 , srv839501274:87 , srv854311398:87 , srv1228556515:88 , srv132979080:88 , srv147307745:88 , srv1610174961:88 , srv2036122239:88 , srv2125043942:88 , srv4128677:88 , srv41557014:88 , srv790234028:88 , srv82916138:88 , srv1449107527:89 , srv1756032925:89 , srv1835066863:89 , srv349886296:89 , srv633128340:89 , srv795773821:89 , srv878017628:89 , srv1421991302:90 , srv202983218:90 , srv296713995:90 , srv432457306:90 , srv547051573:90 , srv819295552:90 , srv1840926564:91 , srv622975780:91 , srv225649486:92 , srv617192622:92 , srv357826191:95 , srv673572888:459 }
2023-11-19 02:55:16,572 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:55:16,573 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1300499582=12, srv1002566505=0, srv45853174=70, srv800014714=91, srv258569830=59, srv1465160612=21, srv1775063387=34, srv147307745=22, srv1093315112=4, srv1866229360=39, srv606293107=80, srv132918371=16, srv727326198=88, srv1090561271=3, srv17515464=32, srv4128677=67, srv839501274=95, srv1152931340=7, srv617192622=81, srv547051573=77, srv1618946733=28, srv41557014=68, srv819295552=93, srv296599171=60, srv979084382=99, srv1533617238=24, srv854311398=97, srv1838832631=36, srv519393859=76, srv210225082=52, srv2133464386=56, srv1058252969=2, srv2089279086=51, srv129915857=11, srv1857176843=38, srv1840926564=37, srv296713995=61, srv1112081858=5, srv349886296=64, srv1663924588=30, srv334097614=63, srv588507424=79, srv1603810695=26, srv132979080=17, srv795773821=90, srv1228556515=9, srv202983218=48, srv2028042582=47, srv1610174961=27, srv1316951789=15, srv2110262852=53, srv573480192=78, srv481833907=73, srv1514400855=23, srv665282157=84, srv1869482791=40, srv1989369085=46, srv815621785=92, srv484756696=74, srv2114294088=54, srv1277744863=10, srv197626510=44, srv465688639=71, srv1756032925=33, srv225649486=57, srv82916138=94, srv1149749254=6, srv1159953757=8, srv1835066863=35, srv2036122239=49, srv2125043942=55, srv1646183935=29, srv718722465=86, srv790234028=89, srv839683146=96, srv878017628=98, srv1314530338=13, srv504133567=75, srv1449107527=20, srv622975780=82, srv357826191=65, srv1599450477=25, srv673572888=85, srv1430938397=19, srv193755933=41, srv1671920660=31, srv1946606357=42, srv257589583=58, srv432457306=69, srv2036188759=50, srv1316331588=14, srv724533251=87, srv103618542=1, srv324767157=62, srv397492349=66, srv633128340=83, srv1969594994=43, srv1986495906=45, srv1421991302=18, srv479775982=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:55:16,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:55:16,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:55:16,586 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:55:16,587 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:55:16,588 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 02:55:16,589 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 02:55:16,590 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 02:55:16,591 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 02:55:16,592 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 02:55:16,593 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 02:55:16,594 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 02:55:16,783 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04150462317448463, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05550098329782828, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04150462317448463, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:16,783 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 02:55:16,783 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17914709734431009, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04150462317448463, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05550098329782828, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04150462317448463, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23991666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 02:55:17,088 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 466 ms to try 5000 different iterations.  Found a solution that moves 2627 regions; Going from a computed imbalance of 0.17914709734431009 to a new imbalance of 0.015729600917768795. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023191541818552857); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03549236809571216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29188888888888886, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023191541818552857); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:17,097 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv357826191:78 , srv1835066863:81 , srv432457306:81 , srv547051573:81 , srv617192622:81 , srv633128340:81 , srv132979080:82 , srv1421991302:82 , srv1840926564:82 , srv202983218:82 , srv225649486:82 , srv1228556515:83 , srv1430938397:83 , srv147307745:83 , srv1646183935:83 , srv1969594994:83 , srv2036122239:83 , srv397492349:83 , srv790234028:83 , srv795773821:83 , srv819295552:83 , srv82916138:83 , srv103618542:84 , srv1093315112:84 , srv1152931340:84 , srv1449107527:84 , srv1465160612:84 , srv1610174961:84 , srv1663924588:84 , srv1756032925:84 , srv1989369085:84 , srv296713995:84 , srv324767157:84 , srv349886296:84 , srv4128677:84 , srv479775982:84 , srv481833907:84 , srv839501274:84 , srv1314530338:85 , srv1618946733:85 , srv2089279086:85 , srv2114294088:85 , srv2125043942:85 , srv41557014:85 , srv573480192:85 , srv588507424:85 , srv724533251:85 , srv854311398:85 , srv878017628:85 , srv979084382:85 , srv1058252969:86 , srv1149749254:86 , srv1300499582:86 , srv1316951789:86 , srv1514400855:86 , srv1599450477:86 , srv1603810695:86 , srv1869482791:86 , srv258569830:86 , srv800014714:86 , srv1002566505:87 , srv1090561271:87 , srv1159953757:87 , srv1316331588:87 , srv1533617238:87 , srv1671920660:87 , srv1857176843:87 , srv1866229360:87 , srv193755933:87 , srv1946606357:87 , srv2110262852:87 , srv465688639:87 , srv665282157:87 , srv718722465:87 , srv815621785:87 , srv17515464:88 , srv1775063387:88 , srv197626510:88 , srv2028042582:88 , srv2133464386:88 , srv334097614:88 , srv484756696:88 , srv727326198:88 , srv1112081858:89 , srv1277744863:89 , srv129915857:89 , srv132918371:89 , srv1986495906:89 , srv210225082:89 , srv504133567:89 , srv622975780:89 , srv839683146:89 , srv257589583:90 , srv519393859:90 , srv296599171:91 , srv45853174:91 , srv2036188759:92 , srv606293107:92 , srv1838832631:93 , srv673572888:521 }
2023-11-19 02:55:17,100 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:55:17,101 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1300499582=12, srv1002566505=0, srv45853174=70, srv800014714=91, srv258569830=59, srv1465160612=21, srv1775063387=34, srv147307745=22, srv1093315112=4, srv1866229360=39, srv606293107=80, srv132918371=16, srv727326198=88, srv1090561271=3, srv17515464=32, srv4128677=67, srv839501274=95, srv1152931340=7, srv617192622=81, srv547051573=77, srv1618946733=28, srv41557014=68, srv819295552=93, srv296599171=60, srv979084382=99, srv1533617238=24, srv854311398=97, srv1838832631=36, srv519393859=76, srv210225082=52, srv2133464386=56, srv1058252969=2, srv2089279086=51, srv129915857=11, srv1857176843=38, srv1840926564=37, srv296713995=61, srv1112081858=5, srv349886296=64, srv1663924588=30, srv334097614=63, srv588507424=79, srv1603810695=26, srv132979080=17, srv795773821=90, srv1228556515=9, srv202983218=48, srv2028042582=47, srv1610174961=27, srv1316951789=15, srv2110262852=53, srv573480192=78, srv481833907=73, srv1514400855=23, srv665282157=84, srv1869482791=40, srv1989369085=46, srv815621785=92, srv484756696=74, srv2114294088=54, srv1277744863=10, srv197626510=44, srv465688639=71, srv1756032925=33, srv225649486=57, srv82916138=94, srv1149749254=6, srv1159953757=8, srv1835066863=35, srv2036122239=49, srv2125043942=55, srv1646183935=29, srv718722465=86, srv790234028=89, srv839683146=96, srv878017628=98, srv1314530338=13, srv504133567=75, srv1449107527=20, srv622975780=82, srv357826191=65, srv1599450477=25, srv673572888=85, srv1430938397=19, srv193755933=41, srv1671920660=31, srv1946606357=42, srv257589583=58, srv432457306=69, srv2036188759=50, srv1316331588=14, srv724533251=87, srv103618542=1, srv324767157=62, srv397492349=66, srv633128340=83, srv1969594994=43, srv1986495906=45, srv1421991302=18, srv479775982=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:55:17,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:55:17,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:55:17,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 02:55:17,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 02:55:17,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 02:55:17,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 02:55:17,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 02:55:17,121 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 02:55:17,305 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023191541818552857); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03549236809571216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023191541818552857); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:17,305 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 02:55:17,305 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015711206270612118, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023191541818552857); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03549236809571216, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023191541818552857); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012666666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.044833333333333336, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 02:55:17,600 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 464 ms to try 5000 different iterations.  Found a solution that moves 1019 regions; Going from a computed imbalance of 0.015711206270612118 to a new imbalance of 0.0010666199324595948. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005543196128553987); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631567); MoveCostFunction : (multiplier=7.0, imbalance=0.11322222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005543196128553987); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:17,602 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv622975780:78 , srv606293107:81 , srv1838832631:82 , srv1775063387:83 , srv1946606357:83 , srv2036188759:83 , srv257589583:83 , srv296599171:83 , srv45853174:83 , srv1277744863:84 , srv132918371:84 , srv1840926564:84 , srv1857176843:84 , srv197626510:84 , srv2133464386:84 , srv225649486:84 , srv296713995:84 , srv349886296:84 , srv357826191:84 , srv504133567:84 , srv519393859:84 , srv617192622:84 , srv819295552:84 , srv878017628:84 , srv1058252969:85 , srv1090561271:85 , srv129915857:85 , srv1316331588:85 , srv1421991302:85 , srv1449107527:85 , srv1533617238:85 , srv1603810695:85 , srv1756032925:85 , srv1866229360:85 , srv193755933:85 , srv202983218:85 , srv2110262852:85 , srv2125043942:85 , srv4128677:85 , srv41557014:85 , srv665282157:85 , srv1112081858:86 , srv1152931340:86 , srv1159953757:86 , srv1300499582:86 , srv1314530338:86 , srv1316951789:86 , srv1465160612:86 , srv1663924588:86 , srv1986495906:86 , srv2028042582:86 , srv2036122239:86 , srv2089279086:86 , srv210225082:86 , srv334097614:86 , srv432457306:86 , srv465688639:86 , srv481833907:86 , srv484756696:86 , srv547051573:86 , srv727326198:86 , srv795773821:86 , srv839501274:86 , srv854311398:86 , srv1002566505:87 , srv103618542:87 , srv1093315112:87 , srv1149749254:87 , srv1228556515:87 , srv132979080:87 , srv147307745:87 , srv1514400855:87 , srv1599450477:87 , srv1610174961:87 , srv1618946733:87 , srv1671920660:87 , srv17515464:87 , srv1835066863:87 , srv1869482791:87 , srv1969594994:87 , srv1989369085:87 , srv258569830:87 , srv479775982:87 , srv573480192:87 , srv588507424:87 , srv633128340:87 , srv718722465:87 , srv790234028:87 , srv800014714:87 , srv815621785:87 , srv82916138:87 , srv839683146:87 , srv979084382:87 , srv1430938397:88 , srv1646183935:88 , srv2114294088:88 , srv324767157:88 , srv397492349:88 , srv724533251:88 , srv673572888:527 }
2023-11-19 02:55:17,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:55:17,606 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1300499582=12, srv1002566505=0, srv45853174=70, srv800014714=91, srv258569830=59, srv1465160612=21, srv1775063387=34, srv147307745=22, srv1093315112=4, srv1866229360=39, srv606293107=80, srv132918371=16, srv727326198=88, srv1090561271=3, srv17515464=32, srv4128677=67, srv839501274=95, srv1152931340=7, srv617192622=81, srv547051573=77, srv1618946733=28, srv41557014=68, srv819295552=93, srv296599171=60, srv979084382=99, srv1533617238=24, srv854311398=97, srv1838832631=36, srv519393859=76, srv210225082=52, srv2133464386=56, srv1058252969=2, srv2089279086=51, srv129915857=11, srv1857176843=38, srv1840926564=37, srv296713995=61, srv1112081858=5, srv349886296=64, srv1663924588=30, srv334097614=63, srv588507424=79, srv1603810695=26, srv132979080=17, srv795773821=90, srv1228556515=9, srv202983218=48, srv2028042582=47, srv1610174961=27, srv1316951789=15, srv2110262852=53, srv573480192=78, srv481833907=73, srv1514400855=23, srv665282157=84, srv1869482791=40, srv1989369085=46, srv815621785=92, srv484756696=74, srv2114294088=54, srv1277744863=10, srv197626510=44, srv465688639=71, srv1756032925=33, srv225649486=57, srv82916138=94, srv1149749254=6, srv1159953757=8, srv1835066863=35, srv2036122239=49, srv2125043942=55, srv1646183935=29, srv718722465=86, srv790234028=89, srv839683146=96, srv878017628=98, srv1314530338=13, srv504133567=75, srv1449107527=20, srv622975780=82, srv357826191=65, srv1599450477=25, srv673572888=85, srv1430938397=19, srv193755933=41, srv1671920660=31, srv1946606357=42, srv257589583=58, srv432457306=69, srv2036188759=50, srv1316331588=14, srv724533251=87, srv103618542=1, srv324767157=62, srv397492349=66, srv633128340=83, srv1969594994=43, srv1986495906=45, srv1421991302=18, srv479775982=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:55:17,618 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:55:17,619 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 02:55:17,620 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 02:55:17,621 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 02:55:17,622 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 02:55:17,623 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 02:55:17,624 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 02:55:17,730 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005543196128553987); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631567); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005543196128553987); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:17,730 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-19 02:55:17,730 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.0010594847419561101, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005543196128553987); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006104220173631567); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005543196128553987); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=6.666666666666666E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0045); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-19 02:55:18,092 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 458 ms to try 5000 different iterations.  Found a solution that moves 263 regions; Going from a computed imbalance of 0.0010594847419561101 to a new imbalance of 1.5867165651055582E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.029222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:18,093 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv839683146:82 , srv1228556515:83 , srv1610174961:83 , srv1618946733:83 , srv1869482791:83 , srv718722465:83 , srv1002566505:84 , srv1314530338:84 , srv132918371:84 , srv132979080:84 , srv1430938397:84 , srv1449107527:84 , srv147307745:84 , srv1514400855:84 , srv1533617238:84 , srv1599450477:84 , srv1603810695:84 , srv1646183935:84 , srv1671920660:84 , srv17515464:84 , srv1756032925:84 , srv193755933:84 , srv197626510:84 , srv1986495906:84 , srv1989369085:84 , srv2036188759:84 , srv210225082:84 , srv2114294088:84 , srv2125043942:84 , srv2133464386:84 , srv225649486:84 , srv258569830:84 , srv296599171:84 , srv296713995:84 , srv334097614:84 , srv397492349:84 , srv41557014:84 , srv479775982:84 , srv547051573:84 , srv573480192:84 , srv622975780:84 , srv665282157:84 , srv724533251:84 , srv790234028:84 , srv795773821:84 , srv800014714:84 , srv815621785:84 , srv854311398:84 , srv878017628:84 , srv103618542:85 , srv1058252969:85 , srv1093315112:85 , srv1112081858:85 , srv1149749254:85 , srv1152931340:85 , srv1159953757:85 , srv1277744863:85 , srv129915857:85 , srv1300499582:85 , srv1316331588:85 , srv1316951789:85 , srv1421991302:85 , srv1465160612:85 , srv1663924588:85 , srv1835066863:85 , srv1838832631:85 , srv1840926564:85 , srv1857176843:85 , srv1866229360:85 , srv1946606357:85 , srv1969594994:85 , srv2028042582:85 , srv2089279086:85 , srv2110262852:85 , srv324767157:85 , srv349886296:85 , srv357826191:85 , srv4128677:85 , srv45853174:85 , srv465688639:85 , srv481833907:85 , srv484756696:85 , srv504133567:85 , srv519393859:85 , srv588507424:85 , srv606293107:85 , srv617192622:85 , srv633128340:85 , srv727326198:85 , srv819295552:85 , srv82916138:85 , srv839501274:85 , srv979084382:85 , srv1090561271:86 , srv1775063387:86 , srv202983218:86 , srv2036122239:86 , srv257589583:86 , srv432457306:86 , srv673572888:635 }
2023-11-19 02:55:18,096 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:55:18,097 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1300499582=12, srv1002566505=0, srv45853174=70, srv800014714=91, srv258569830=59, srv1465160612=21, srv1775063387=34, srv147307745=22, srv1093315112=4, srv1866229360=39, srv606293107=80, srv132918371=16, srv727326198=88, srv1090561271=3, srv17515464=32, srv4128677=67, srv839501274=95, srv1152931340=7, srv617192622=81, srv547051573=77, srv1618946733=28, srv41557014=68, srv819295552=93, srv296599171=60, srv979084382=99, srv1533617238=24, srv854311398=97, srv1838832631=36, srv519393859=76, srv210225082=52, srv2133464386=56, srv1058252969=2, srv2089279086=51, srv129915857=11, srv1857176843=38, srv1840926564=37, srv296713995=61, srv1112081858=5, srv349886296=64, srv1663924588=30, srv334097614=63, srv588507424=79, srv1603810695=26, srv132979080=17, srv795773821=90, srv1228556515=9, srv202983218=48, srv2028042582=47, srv1610174961=27, srv1316951789=15, srv2110262852=53, srv573480192=78, srv481833907=73, srv1514400855=23, srv665282157=84, srv1869482791=40, srv1989369085=46, srv815621785=92, srv484756696=74, srv2114294088=54, srv1277744863=10, srv197626510=44, srv465688639=71, srv1756032925=33, srv225649486=57, srv82916138=94, srv1149749254=6, srv1159953757=8, srv1835066863=35, srv2036122239=49, srv2125043942=55, srv1646183935=29, srv718722465=86, srv790234028=89, srv839683146=96, srv878017628=98, srv1314530338=13, srv504133567=75, srv1449107527=20, srv622975780=82, srv357826191=65, srv1599450477=25, srv673572888=85, srv1430938397=19, srv193755933=41, srv1671920660=31, srv1946606357=42, srv257589583=58, srv432457306=69, srv2036188759=50, srv1316331588=14, srv724533251=87, srv103618542=1, srv324767157=62, srv397492349=66, srv633128340=83, srv1969594994=43, srv1986495906=45, srv1421991302=18, srv479775982=72} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:55:18,108 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:55:18,109 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-19 02:55:18,110 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-19 02:55:18,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-19 02:55:18,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-19 02:55:18,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-19 02:55:18,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-19 02:55:18,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-19 02:55:18,220 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.4025600290489888E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0026378918903435544); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:18,220 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv839683146:82 , srv1228556515:83 , srv1610174961:83 , srv1618946733:83 , srv1869482791:83 , srv718722465:83 , srv1002566505:84 , srv1314530338:84 , srv132918371:84 , srv132979080:84 , srv1430938397:84 , srv1449107527:84 , srv147307745:84 , srv1514400855:84 , srv1533617238:84 , srv1599450477:84 , srv1603810695:84 , srv1646183935:84 , srv1671920660:84 , srv17515464:84 , srv1756032925:84 , srv193755933:84 , srv197626510:84 , srv1986495906:84 , srv1989369085:84 , srv2036188759:84 , srv210225082:84 , srv2114294088:84 , srv2125043942:84 , srv2133464386:84 , srv225649486:84 , srv258569830:84 , srv296599171:84 , srv296713995:84 , srv334097614:84 , srv397492349:84 , srv41557014:84 , srv479775982:84 , srv547051573:84 , srv573480192:84 , srv622975780:84 , srv665282157:84 , srv724533251:84 , srv790234028:84 , srv795773821:84 , srv800014714:84 , srv815621785:84 , srv854311398:84 , srv878017628:84 , srv103618542:85 , srv1058252969:85 , srv1093315112:85 , srv1112081858:85 , srv1149749254:85 , srv1152931340:85 , srv1159953757:85 , srv1277744863:85 , srv129915857:85 , srv1300499582:85 , srv1316331588:85 , srv1316951789:85 , srv1421991302:85 , srv1465160612:85 , srv1663924588:85 , srv1835066863:85 , srv1838832631:85 , srv1840926564:85 , srv1857176843:85 , srv1866229360:85 , srv1946606357:85 , srv1969594994:85 , srv2028042582:85 , srv2089279086:85 , srv2110262852:85 , srv324767157:85 , srv349886296:85 , srv357826191:85 , srv4128677:85 , srv45853174:85 , srv465688639:85 , srv481833907:85 , srv484756696:85 , srv504133567:85 , srv519393859:85 , srv588507424:85 , srv606293107:85 , srv617192622:85 , srv633128340:85 , srv727326198:85 , srv819295552:85 , srv82916138:85 , srv839501274:85 , srv979084382:85 , srv1090561271:86 , srv1775063387:86 , srv202983218:86 , srv2036122239:86 , srv257589583:86 , srv432457306:86 , srv673572888:635 }
2023-11-19 02:55:18,244 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=100 (was 100), ProcessCount=170 (was 177), AvailableMemoryMB=18168 (was 18653)
2023-11-19 02:55:18,251 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=100, ProcessCount=170, AvailableMemoryMB=18166
2023-11-19 02:55:18,251 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-19 02:55:18,252 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-19 02:55:18,259 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1141640546:84 , srv1174771202:84 , srv1350871493:84 , srv1364875357:84 , srv1384856442:84 , srv141212485:84 , srv1415691900:84 , srv14338664:84 , srv1451129243:84 , srv1460935707:84 , srv1694953082:84 , srv1727997306:84 , srv1731914173:84 , srv1788706814:84 , srv1874891684:84 , srv1907719881:84 , srv1922720534:84 , srv2024230695:84 , srv2027961880:84 , srv2085484850:84 , srv2096034512:84 , srv21375401:84 , srv217665207:84 , srv347634137:84 , srv569108563:84 , srv608905412:84 , srv661422783:84 , srv743525747:84 , srv860750254:84 , srv2038085549:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-19 02:55:18,260 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:55:18,261 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1788706814=13, srv217665207=23, srv1874891684=14, srv661422783=27, srv1731914173=12, srv1174771202=1, srv1451129243=8, srv1415691900=6, srv1364875357=3, srv21375401=22, srv141212485=5, srv2038085549=19, srv1907719881=15, srv347634137=24, srv2024230695=17, srv1384856442=4, srv2027961880=18, srv1694953082=10, srv2096034512=21, srv860750254=29, srv1141640546=0, srv1922720534=16, srv2085484850=20, srv569108563=25, srv1727997306=11, srv1350871493=2, srv743525747=28, srv1460935707=9, srv608905412=26, srv14338664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:55:18,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:55:18,270 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:55:18,271 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-19 02:55:18,284 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0761221200246964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:55:18,284 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912432901578863, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0761221200246964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-19 02:57:18,273 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5380677 different iterations.  Found a solution that moves 1881 regions; Going from a computed imbalance of 0.9912432901578863 to a new imbalance of 4.390347836785893E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6966666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:57:18,280 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1141640546:90 , srv1174771202:90 , srv1350871493:90 , srv1364875357:90 , srv1384856442:90 , srv141212485:90 , srv1415691900:90 , srv14338664:90 , srv1451129243:90 , srv1460935707:90 , srv1694953082:90 , srv1727997306:90 , srv1731914173:90 , srv1788706814:90 , srv1874891684:90 , srv1907719881:90 , srv1922720534:90 , srv2024230695:90 , srv2027961880:90 , srv2038085549:90 , srv2085484850:90 , srv2096034512:90 , srv21375401:90 , srv217665207:90 , srv347634137:90 , srv569108563:90 , srv608905412:90 , srv661422783:90 , srv743525747:90 , srv860750254:90 }
2023-11-19 02:57:18,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-19 02:57:18,281 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1788706814=13, srv217665207=23, srv1874891684=14, srv661422783=27, srv1731914173=12, srv1174771202=1, srv1451129243=8, srv1415691900=6, srv1364875357=3, srv21375401=22, srv141212485=5, srv2038085549=19, srv1907719881=15, srv347634137=24, srv2024230695=17, srv1384856442=4, srv2027961880=18, srv1694953082=10, srv2096034512=21, srv860750254=29, srv1141640546=0, srv1922720534=16, srv2085484850=20, srv569108563=25, srv1727997306=11, srv1350871493=2, srv743525747=28, srv1460935707=9, srv608905412=26, srv14338664=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-19 02:57:18,285 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-19 02:57:18,286 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-19 02:57:18,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-19 02:57:18,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-19 02:57:18,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-19 02:57:18,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-19 02:57:18,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-19 02:57:18,287 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-19 02:57:18,300 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-19 02:57:18,300 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1141640546:90 , srv1174771202:90 , srv1350871493:90 , srv1364875357:90 , srv1384856442:90 , srv141212485:90 , srv1415691900:90 , srv14338664:90 , srv1451129243:90 , srv1460935707:90 , srv1694953082:90 , srv1727997306:90 , srv1731914173:90 , srv1788706814:90 , srv1874891684:90 , srv1907719881:90 , srv1922720534:90 , srv2024230695:90 , srv2027961880:90 , srv2038085549:90 , srv2085484850:90 , srv2096034512:90 , srv21375401:90 , srv217665207:90 , srv347634137:90 , srv569108563:90 , srv608905412:90 , srv661422783:90 , srv743525747:90 , srv860750254:90 }
2023-11-19 02:57:18,317 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=172 (was 100) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 170), AvailableMemoryMB=18139 (was 18166)