2023-11-08 22:58:59,325 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-08 22:58:59,591 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-08 22:58:59,714 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-08 22:58:59,777 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-08 22:58:59,783 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-08 22:58:59,795 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=375, ProcessCount=166, AvailableMemoryMB=6978
2023-11-08 22:58:59,796 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-08 22:58:59,797 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-08 22:58:59,911 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100497238:84 , srv1008014144:84 , srv1009048316:84 , srv1009259880:84 , srv1018496621:84 , srv1023182584:84 , srv1043258802:84 , srv105267111:84 , srv1084492301:84 , srv1090029262:84 , srv1092111886:84 , srv1104343064:84 , srv1107910630:84 , srv117108809:84 , srv1176407167:84 , srv1180178134:84 , srv1225873370:84 , srv1292313859:84 , srv1311672615:84 , srv1390842652:84 , srv1398726863:84 , srv1420911307:84 , srv1458017686:84 , srv1461728394:84 , srv1476598138:84 , srv1480310794:84 , srv1509560974:84 , srv1521042310:84 , srv153060544:84 , srv1537980075:84 , srv1553207487:84 , srv1584263858:84 , srv1616058416:84 , srv162095538:84 , srv1622610644:84 , srv1641245599:84 , srv165983488:84 , srv1673197585:84 , srv1697625864:84 , srv1713932214:84 , srv1750800702:84 , srv1750882245:84 , srv1757703812:84 , srv1757880196:84 , srv1780243488:84 , srv17851264:84 , srv1797762391:84 , srv1814046737:84 , srv1835866694:84 , srv1924984302:84 , srv2005477710:84 , srv2017690895:84 , srv2029403673:84 , srv2052363143:84 , srv2096148912:84 , srv2121175027:84 , srv2122699823:84 , srv2127690119:84 , srv218119069:84 , srv253060225:84 , srv284247200:84 , srv326418616:84 , srv332878008:84 , srv334108494:84 , srv351337883:84 , srv351685070:84 , srv384740078:84 , srv39413807:84 , srv403699611:84 , srv432204217:84 , srv432777796:84 , srv441410298:84 , srv457460314:84 , srv470028780:84 , srv512217200:84 , srv586332404:84 , srv599190137:84 , srv60096818:84 , srv620228540:84 , srv635740612:84 , srv645193822:84 , srv652530187:84 , srv653349079:84 , srv694277461:84 , srv725738440:84 , srv727887941:84 , srv732558905:84 , srv750540436:84 , srv787559780:84 , srv808541429:84 , srv819554675:84 , srv854462171:84 , srv887464687:84 , srv922772728:84 , srv93120102:84 , srv953272379:84 , srv955537589:84 , srv963824517:84 , srv993828097:84 , srv391471813:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-08 22:58:59,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:58:59,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv963824517=98, srv351685070=65, srv1814046737=47, srv1616058416=32, srv1008014144=1, srv1023182584=5, srv620228540=79, srv1084492301=8, srv1461728394=23, srv1622610644=34, srv332878008=62, srv2121175027=55, srv922772728=94, srv432777796=71, srv1697625864=38, srv1225873370=16, srv2029403673=52, srv326418616=61, srv1180178134=15, srv1713932214=39, srv653349079=83, srv953272379=96, srv1673197585=37, srv645193822=81, srv635740612=80, srv1476598138=24, srv162095538=33, srv1420911307=21, srv253060225=59, srv1458017686=22, srv1924984302=49, srv1107910630=12, srv284247200=60, srv694277461=84, srv1780243488=44, srv457460314=73, srv750540436=88, srv1292313859=17, srv1537980075=29, srv1584263858=31, srv165983488=36, srv100497238=0, srv599190137=77, srv1043258802=6, srv334108494=63, srv351337883=64, srv787559780=89, srv1176407167=14, srv1553207487=30, srv512217200=75, srv652530187=82, srv2096148912=54, srv218119069=58, srv819554675=91, srv2127690119=57, srv808541429=90, srv93120102=95, srv1480310794=25, srv1835866694=48, srv153060544=28, srv1090029262=9, srv1104343064=11, srv887464687=93, srv1009259880=3, srv384740078=66, srv1009048316=2, srv1311672615=18, srv17851264=45, srv39413807=68, srv1757880196=43, srv586332404=76, srv2052363143=53, srv2122699823=56, srv1750800702=40, srv441410298=72, srv105267111=7, srv1757703812=42, srv1750882245=41, srv1398726863=20, srv432204217=70, srv732558905=87, srv1797762391=46, srv470028780=74, srv955537589=97, srv727887941=86, srv60096818=78, srv1390842652=19, srv993828097=99, srv117108809=13, srv725738440=85, srv1641245599=35, srv391471813=67, srv2005477710=50, srv1092111886=10, srv1018496621=4, srv1521042310=27, srv1509560974=26, srv2017690895=51, srv854462171=92, srv403699611=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:00,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:00,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:00,023 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:00,024 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:00,025 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:00,026 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-08 22:59:00,027 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-08 22:59:00,028 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-08 22:59:00,029 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-08 22:59:00,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-08 22:59:00,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-08 22:59:00,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-08 22:59:00,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-08 22:59:00,030 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-08 22:59:00,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-08 22:59:00,032 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-08 22:59:00,033 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-08 22:59:00,034 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-08 22:59:00,035 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-08 22:59:00,036 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-08 22:59:00,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-08 22:59:00,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-08 22:59:00,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-08 22:59:00,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-08 22:59:00,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-08 22:59:00,037 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-08 22:59:00,038 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-08 22:59:00,039 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-08 22:59:00,223 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:00,224 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-08 22:59:00,224 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-08 22:59:00,493 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 428 ms to try 5000 different iterations.  Found a solution that moves 4723 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17259720450308594. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04061474241465964, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057354705905987256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5247777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04061474241465964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16766666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23508333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:00,502 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1009048316:82 , srv162095538:82 , srv218119069:82 , srv403699611:82 , srv635740612:82 , srv1084492301:83 , srv153060544:83 , srv1750882245:83 , srv1757880196:83 , srv2127690119:83 , srv39413807:83 , srv457460314:83 , srv586332404:83 , srv60096818:83 , srv725738440:83 , srv922772728:83 , srv953272379:83 , srv1311672615:84 , srv17851264:84 , srv2122699823:84 , srv332878008:84 , srv470028780:84 , srv645193822:84 , srv652530187:84 , srv727887941:84 , srv787559780:84 , srv808541429:84 , srv1107910630:85 , srv117108809:85 , srv1757703812:85 , srv1814046737:85 , srv2052363143:85 , srv351685070:85 , srv750540436:85 , srv819554675:85 , srv854462171:85 , srv93120102:85 , srv1008014144:86 , srv1018496621:86 , srv105267111:86 , srv1090029262:86 , srv1092111886:86 , srv1176407167:86 , srv1398726863:86 , srv1458017686:86 , srv1476598138:86 , srv1553207487:86 , srv1622610644:86 , srv1641245599:86 , srv1673197585:86 , srv1750800702:86 , srv1835866694:86 , srv326418616:86 , srv887464687:86 , srv1180178134:87 , srv1521042310:87 , srv165983488:87 , srv1697625864:87 , srv1713932214:87 , srv2096148912:87 , srv334108494:87 , srv599190137:87 , srv694277461:87 , srv732558905:87 , srv955537589:87 , srv1043258802:88 , srv1225873370:88 , srv1480310794:88 , srv1509560974:88 , srv2005477710:88 , srv2017690895:88 , srv2121175027:88 , srv351337883:88 , srv620228540:88 , srv653349079:88 , srv1009259880:89 , srv1023182584:89 , srv1292313859:89 , srv1390842652:89 , srv1461728394:89 , srv1537980075:89 , srv1780243488:89 , srv1797762391:89 , srv2029403673:89 , srv284247200:89 , srv384740078:89 , srv432204217:89 , srv512217200:89 , srv100497238:90 , srv1420911307:90 , srv1616058416:90 , srv1924984302:90 , srv253060225:90 , srv441410298:90 , srv963824517:90 , srv1104343064:91 , srv1584263858:91 , srv432777796:92 , srv993828097:93 , srv391471813:451 }
2023-11-08 22:59:00,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:59:00,508 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv963824517=98, srv351685070=65, srv1814046737=47, srv1616058416=32, srv1008014144=1, srv1023182584=5, srv620228540=79, srv1084492301=8, srv1461728394=23, srv1622610644=34, srv332878008=62, srv2121175027=55, srv922772728=94, srv432777796=71, srv1697625864=38, srv1225873370=16, srv2029403673=52, srv326418616=61, srv1180178134=15, srv1713932214=39, srv653349079=83, srv953272379=96, srv1673197585=37, srv645193822=81, srv635740612=80, srv1476598138=24, srv162095538=33, srv1420911307=21, srv253060225=59, srv1458017686=22, srv1924984302=49, srv1107910630=12, srv284247200=60, srv694277461=84, srv1780243488=44, srv457460314=73, srv750540436=88, srv1292313859=17, srv1537980075=29, srv1584263858=31, srv165983488=36, srv100497238=0, srv599190137=77, srv1043258802=6, srv334108494=63, srv351337883=64, srv787559780=89, srv1176407167=14, srv1553207487=30, srv512217200=75, srv652530187=82, srv2096148912=54, srv218119069=58, srv819554675=91, srv2127690119=57, srv808541429=90, srv93120102=95, srv1480310794=25, srv1835866694=48, srv153060544=28, srv1090029262=9, srv1104343064=11, srv887464687=93, srv1009259880=3, srv384740078=66, srv1009048316=2, srv1311672615=18, srv17851264=45, srv39413807=68, srv1757880196=43, srv586332404=76, srv2052363143=53, srv2122699823=56, srv1750800702=40, srv441410298=72, srv105267111=7, srv1757703812=42, srv1750882245=41, srv1398726863=20, srv432204217=70, srv732558905=87, srv1797762391=46, srv470028780=74, srv955537589=97, srv727887941=86, srv60096818=78, srv1390842652=19, srv993828097=99, srv117108809=13, srv725738440=85, srv1641245599=35, srv391471813=67, srv2005477710=50, srv1092111886=10, srv1018496621=4, srv1521042310=27, srv1509560974=26, srv2017690895=51, srv854462171=92, srv403699611=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:00,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:00,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:00,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:00,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:00,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:00,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:00,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:00,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:00,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:00,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:00,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:00,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-08 22:59:00,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-08 22:59:00,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-08 22:59:00,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-08 22:59:00,530 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-08 22:59:00,531 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-08 22:59:00,532 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-08 22:59:00,533 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-08 22:59:00,652 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04061474241465964, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057354705905987256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04061474241465964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16766666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23508333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:00,652 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-08 22:59:00,652 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.172564133350242, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04061474241465964, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.057354705905987256, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04061474241465964, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.16766666666666666, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.23508333333333334, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-08 22:59:00,836 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 278 ms to try 5000 different iterations.  Found a solution that moves 2638 regions; Going from a computed imbalance of 0.172564133350242 to a new imbalance of 0.015241072087880433. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02173999792236758); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03683310861183486, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2931111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02173999792236758); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:00,845 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv432777796:80 , srv993828097:80 , srv1480310794:81 , srv1924984302:81 , srv1009259880:82 , srv1420911307:82 , srv1461728394:82 , srv1616058416:82 , srv1780243488:82 , srv441410298:82 , srv100497238:83 , srv1104343064:83 , srv1292313859:83 , srv1584263858:83 , srv2029403673:83 , srv2121175027:83 , srv284247200:83 , srv432204217:83 , srv512217200:83 , srv694277461:83 , srv105267111:84 , srv1090029262:84 , srv1092111886:84 , srv1176407167:84 , srv1225873370:84 , srv1390842652:84 , srv1509560974:84 , srv1521042310:84 , srv1673197585:84 , srv1697625864:84 , srv1757703812:84 , srv2017690895:84 , srv253060225:84 , srv334108494:84 , srv351337883:84 , srv599190137:84 , srv653349079:84 , srv1023182584:85 , srv1398726863:85 , srv1641245599:85 , srv165983488:85 , srv1713932214:85 , srv1750800702:85 , srv2005477710:85 , srv620228540:85 , srv750540436:85 , srv955537589:85 , srv963824517:85 , srv1008014144:86 , srv1107910630:86 , srv117108809:86 , srv1311672615:86 , srv1458017686:86 , srv1537980075:86 , srv1622610644:86 , srv1797762391:86 , srv2052363143:86 , srv2096148912:86 , srv2127690119:86 , srv326418616:86 , srv384740078:86 , srv457460314:86 , srv787559780:86 , srv808541429:86 , srv93120102:86 , srv1043258802:87 , srv1476598138:87 , srv1553207487:87 , srv1750882245:87 , srv17851264:87 , srv1835866694:87 , srv351685070:87 , srv586332404:87 , srv819554675:87 , srv887464687:87 , srv1018496621:88 , srv162095538:88 , srv332878008:88 , srv645193822:88 , srv854462171:88 , srv922772728:88 , srv153060544:89 , srv1757880196:89 , srv2122699823:89 , srv470028780:89 , srv60096818:89 , srv635740612:89 , srv727887941:89 , srv1009048316:90 , srv1084492301:90 , srv1180178134:90 , srv218119069:90 , srv39413807:90 , srv403699611:90 , srv732558905:90 , srv953272379:90 , srv1814046737:91 , srv725738440:91 , srv652530187:92 , srv391471813:516 }
2023-11-08 22:59:00,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:59:00,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv963824517=98, srv351685070=65, srv1814046737=47, srv1616058416=32, srv1008014144=1, srv1023182584=5, srv620228540=79, srv1084492301=8, srv1461728394=23, srv1622610644=34, srv332878008=62, srv2121175027=55, srv922772728=94, srv432777796=71, srv1697625864=38, srv1225873370=16, srv2029403673=52, srv326418616=61, srv1180178134=15, srv1713932214=39, srv653349079=83, srv953272379=96, srv1673197585=37, srv645193822=81, srv635740612=80, srv1476598138=24, srv162095538=33, srv1420911307=21, srv253060225=59, srv1458017686=22, srv1924984302=49, srv1107910630=12, srv284247200=60, srv694277461=84, srv1780243488=44, srv457460314=73, srv750540436=88, srv1292313859=17, srv1537980075=29, srv1584263858=31, srv165983488=36, srv100497238=0, srv599190137=77, srv1043258802=6, srv334108494=63, srv351337883=64, srv787559780=89, srv1176407167=14, srv1553207487=30, srv512217200=75, srv652530187=82, srv2096148912=54, srv218119069=58, srv819554675=91, srv2127690119=57, srv808541429=90, srv93120102=95, srv1480310794=25, srv1835866694=48, srv153060544=28, srv1090029262=9, srv1104343064=11, srv887464687=93, srv1009259880=3, srv384740078=66, srv1009048316=2, srv1311672615=18, srv17851264=45, srv39413807=68, srv1757880196=43, srv586332404=76, srv2052363143=53, srv2122699823=56, srv1750800702=40, srv441410298=72, srv105267111=7, srv1757703812=42, srv1750882245=41, srv1398726863=20, srv432204217=70, srv732558905=87, srv1797762391=46, srv470028780=74, srv955537589=97, srv727887941=86, srv60096818=78, srv1390842652=19, srv993828097=99, srv117108809=13, srv725738440=85, srv1641245599=35, srv391471813=67, srv2005477710=50, srv1092111886=10, srv1018496621=4, srv1521042310=27, srv1509560974=26, srv2017690895=51, srv854462171=92, srv403699611=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:00,869 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:00,870 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-08 22:59:00,871 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-08 22:59:00,872 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-08 22:59:00,873 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-08 22:59:00,874 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-08 22:59:00,875 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-08 22:59:00,876 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-08 22:59:00,877 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-08 22:59:00,878 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-08 22:59:00,879 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-08 22:59:00,975 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02173999792236758); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03683310861183486, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02173999792236758); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:00,975 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-08 22:59:00,976 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.015222600417077497, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02173999792236758); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03683310861183486, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02173999792236758); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.012416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.041916666666666665, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-08 22:59:01,145 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 252 ms to try 5000 different iterations.  Found a solution that moves 989 regions; Going from a computed imbalance of 0.015222600417077497 to a new imbalance of 5.193079757503613E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006997051024024128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01134105500469549); MoveCostFunction : (multiplier=7.0, imbalance=0.10988888888888888, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006997051024024128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,146 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1180178134:81 , srv1814046737:81 , srv652530187:81 , srv732558905:81 , srv1043258802:82 , srv963824517:82 , srv1537980075:83 , srv1584263858:83 , srv253060225:83 , srv384740078:83 , srv100497238:84 , srv1018496621:84 , srv1084492301:84 , srv1390842652:84 , srv1476598138:84 , srv1797762391:84 , srv1835866694:84 , srv2005477710:84 , srv2096148912:84 , srv2122699823:84 , srv39413807:84 , srv470028780:84 , srv725738440:84 , srv953272379:84 , srv993828097:84 , srv1008014144:85 , srv1104343064:85 , srv117108809:85 , srv1420911307:85 , srv1458017686:85 , srv1509560974:85 , srv1553207487:85 , srv1616058416:85 , srv1622610644:85 , srv165983488:85 , srv1713932214:85 , srv1757880196:85 , srv2017690895:85 , srv2029403673:85 , srv218119069:85 , srv326418616:85 , srv351337883:85 , srv351685070:85 , srv403699611:85 , srv432204217:85 , srv441410298:85 , srv512217200:85 , srv60096818:85 , srv620228540:85 , srv727887941:85 , srv819554675:85 , srv854462171:85 , srv887464687:85 , srv955537589:85 , srv1009048316:86 , srv1009259880:86 , srv1023182584:86 , srv1090029262:86 , srv1107910630:86 , srv1225873370:86 , srv1292313859:86 , srv1461728394:86 , srv1521042310:86 , srv153060544:86 , srv1641245599:86 , srv1697625864:86 , srv1750800702:86 , srv1750882245:86 , srv1780243488:86 , srv17851264:86 , srv1924984302:86 , srv2121175027:86 , srv332878008:86 , srv432777796:86 , srv599190137:86 , srv635740612:86 , srv645193822:86 , srv653349079:86 , srv922772728:86 , srv93120102:86 , srv1398726863:87 , srv162095538:87 , srv2052363143:87 , srv284247200:87 , srv334108494:87 , srv586332404:87 , srv694277461:87 , srv750540436:87 , srv787559780:87 , srv808541429:87 , srv105267111:88 , srv1092111886:88 , srv1311672615:88 , srv1480310794:88 , srv1673197585:88 , srv1176407167:89 , srv1757703812:89 , srv2127690119:89 , srv457460314:89 , srv391471813:553 }
2023-11-08 22:59:01,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:59:01,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv963824517=98, srv351685070=65, srv1814046737=47, srv1616058416=32, srv1008014144=1, srv1023182584=5, srv620228540=79, srv1084492301=8, srv1461728394=23, srv1622610644=34, srv332878008=62, srv2121175027=55, srv922772728=94, srv432777796=71, srv1697625864=38, srv1225873370=16, srv2029403673=52, srv326418616=61, srv1180178134=15, srv1713932214=39, srv653349079=83, srv953272379=96, srv1673197585=37, srv645193822=81, srv635740612=80, srv1476598138=24, srv162095538=33, srv1420911307=21, srv253060225=59, srv1458017686=22, srv1924984302=49, srv1107910630=12, srv284247200=60, srv694277461=84, srv1780243488=44, srv457460314=73, srv750540436=88, srv1292313859=17, srv1537980075=29, srv1584263858=31, srv165983488=36, srv100497238=0, srv599190137=77, srv1043258802=6, srv334108494=63, srv351337883=64, srv787559780=89, srv1176407167=14, srv1553207487=30, srv512217200=75, srv652530187=82, srv2096148912=54, srv218119069=58, srv819554675=91, srv2127690119=57, srv808541429=90, srv93120102=95, srv1480310794=25, srv1835866694=48, srv153060544=28, srv1090029262=9, srv1104343064=11, srv887464687=93, srv1009259880=3, srv384740078=66, srv1009048316=2, srv1311672615=18, srv17851264=45, srv39413807=68, srv1757880196=43, srv586332404=76, srv2052363143=53, srv2122699823=56, srv1750800702=40, srv441410298=72, srv105267111=7, srv1757703812=42, srv1750882245=41, srv1398726863=20, srv432204217=70, srv732558905=87, srv1797762391=46, srv470028780=74, srv955537589=97, srv727887941=86, srv60096818=78, srv1390842652=19, srv993828097=99, srv117108809=13, srv725738440=85, srv1641245599=35, srv391471813=67, srv2005477710=50, srv1092111886=10, srv1018496621=4, srv1521042310=27, srv1509560974=26, srv2017690895=51, srv854462171=92, srv403699611=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:01,160 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:01,161 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-08 22:59:01,162 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-08 22:59:01,163 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-08 22:59:01,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-08 22:59:01,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-08 22:59:01,166 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-08 22:59:01,257 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006997051024024128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01134105500469549); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006997051024024128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,257 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-08 22:59:01,258 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.12382849736675E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006997051024024128); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.01134105500469549); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006997051024024128); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-08 22:59:01,494 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 317 ms to try 5000 different iterations.  Found a solution that moves 301 regions; Going from a computed imbalance of 5.12382849736675E-4 to a new imbalance of 2.7655390192530222E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.03344444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,495 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1023182584:82 , srv1176407167:83 , srv1797762391:83 , srv2127690119:83 , srv284247200:83 , srv1009048316:84 , srv105267111:84 , srv1104343064:84 , srv1180178134:84 , srv1225873370:84 , srv1292313859:84 , srv1311672615:84 , srv1390842652:84 , srv1398726863:84 , srv153060544:84 , srv1537980075:84 , srv1673197585:84 , srv1757703812:84 , srv1780243488:84 , srv2052363143:84 , srv253060225:84 , srv332878008:84 , srv334108494:84 , srv432777796:84 , srv457460314:84 , srv620228540:84 , srv645193822:84 , srv653349079:84 , srv694277461:84 , srv725738440:84 , srv727887941:84 , srv732558905:84 , srv808541429:84 , srv854462171:84 , srv887464687:84 , srv100497238:85 , srv1008014144:85 , srv1009259880:85 , srv1018496621:85 , srv1043258802:85 , srv1084492301:85 , srv1090029262:85 , srv1092111886:85 , srv1107910630:85 , srv1420911307:85 , srv1458017686:85 , srv1461728394:85 , srv1476598138:85 , srv1480310794:85 , srv1509560974:85 , srv1521042310:85 , srv1553207487:85 , srv1584263858:85 , srv1616058416:85 , srv162095538:85 , srv1622610644:85 , srv1641245599:85 , srv165983488:85 , srv1697625864:85 , srv1750800702:85 , srv1757880196:85 , srv17851264:85 , srv1814046737:85 , srv1835866694:85 , srv1924984302:85 , srv2005477710:85 , srv2017690895:85 , srv2029403673:85 , srv2096148912:85 , srv2121175027:85 , srv2122699823:85 , srv218119069:85 , srv326418616:85 , srv351337883:85 , srv351685070:85 , srv384740078:85 , srv39413807:85 , srv403699611:85 , srv432204217:85 , srv441410298:85 , srv470028780:85 , srv512217200:85 , srv586332404:85 , srv599190137:85 , srv60096818:85 , srv635740612:85 , srv652530187:85 , srv750540436:85 , srv787559780:85 , srv819554675:85 , srv922772728:85 , srv93120102:85 , srv953272379:85 , srv963824517:85 , srv993828097:85 , srv117108809:86 , srv1713932214:86 , srv1750882245:86 , srv955537589:86 , srv391471813:622 }
2023-11-08 22:59:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:59:01,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv963824517=98, srv351685070=65, srv1814046737=47, srv1616058416=32, srv1008014144=1, srv1023182584=5, srv620228540=79, srv1084492301=8, srv1461728394=23, srv1622610644=34, srv332878008=62, srv2121175027=55, srv922772728=94, srv432777796=71, srv1697625864=38, srv1225873370=16, srv2029403673=52, srv326418616=61, srv1180178134=15, srv1713932214=39, srv653349079=83, srv953272379=96, srv1673197585=37, srv645193822=81, srv635740612=80, srv1476598138=24, srv162095538=33, srv1420911307=21, srv253060225=59, srv1458017686=22, srv1924984302=49, srv1107910630=12, srv284247200=60, srv694277461=84, srv1780243488=44, srv457460314=73, srv750540436=88, srv1292313859=17, srv1537980075=29, srv1584263858=31, srv165983488=36, srv100497238=0, srv599190137=77, srv1043258802=6, srv334108494=63, srv351337883=64, srv787559780=89, srv1176407167=14, srv1553207487=30, srv512217200=75, srv652530187=82, srv2096148912=54, srv218119069=58, srv819554675=91, srv2127690119=57, srv808541429=90, srv93120102=95, srv1480310794=25, srv1835866694=48, srv153060544=28, srv1090029262=9, srv1104343064=11, srv887464687=93, srv1009259880=3, srv384740078=66, srv1009048316=2, srv1311672615=18, srv17851264=45, srv39413807=68, srv1757880196=43, srv586332404=76, srv2052363143=53, srv2122699823=56, srv1750800702=40, srv441410298=72, srv105267111=7, srv1757703812=42, srv1750882245=41, srv1398726863=20, srv432204217=70, srv732558905=87, srv1797762391=46, srv470028780=74, srv955537589=97, srv727887941=86, srv60096818=78, srv1390842652=19, srv993828097=99, srv117108809=13, srv725738440=85, srv1641245599=35, srv391471813=67, srv2005477710=50, srv1092111886=10, srv1018496621=4, srv1521042310=27, srv1509560974=26, srv2017690895=51, srv854462171=92, srv403699611=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:01,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:01,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:01,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:01,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:01,514 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:01,515 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:01,516 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-08 22:59:01,517 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-08 22:59:01,518 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-08 22:59:01,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-08 22:59:01,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-08 22:59:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-08 22:59:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-08 22:59:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-08 22:59:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-08 22:59:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-08 22:59:01,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-08 22:59:01,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-08 22:59:01,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-08 22:59:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-08 22:59:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-08 22:59:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-08 22:59:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-08 22:59:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-08 22:59:01,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-08 22:59:01,606 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,606 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-08 22:59:01,607 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.5547743144886592E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-08 22:59:01,771 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 240 ms to try 5000 different iterations.  Found a solution that moves 31 regions; Going from a computed imbalance of 2.5547743144886592E-5 to a new imbalance of 6.615052311728012E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0034444444444444444); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,771 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1018496621:83 , srv1092111886:83 , srv1104343064:83 , srv1553207487:83 , srv1713932214:83 , srv384740078:83 , srv955537589:83 , srv100497238:84 , srv1008014144:84 , srv1009048316:84 , srv1009259880:84 , srv1023182584:84 , srv1043258802:84 , srv105267111:84 , srv1084492301:84 , srv1107910630:84 , srv117108809:84 , srv1176407167:84 , srv1180178134:84 , srv1225873370:84 , srv1292313859:84 , srv1311672615:84 , srv1398726863:84 , srv1420911307:84 , srv1458017686:84 , srv1461728394:84 , srv1476598138:84 , srv1480310794:84 , srv1509560974:84 , srv1521042310:84 , srv153060544:84 , srv1537980075:84 , srv1584263858:84 , srv1616058416:84 , srv162095538:84 , srv1622610644:84 , srv1641245599:84 , srv165983488:84 , srv1673197585:84 , srv1697625864:84 , srv1750800702:84 , srv1750882245:84 , srv1757703812:84 , srv1757880196:84 , srv17851264:84 , srv1797762391:84 , srv1814046737:84 , srv1835866694:84 , srv1924984302:84 , srv2005477710:84 , srv2017690895:84 , srv2029403673:84 , srv2052363143:84 , srv2096148912:84 , srv2121175027:84 , srv2122699823:84 , srv218119069:84 , srv284247200:84 , srv326418616:84 , srv332878008:84 , srv334108494:84 , srv351337883:84 , srv351685070:84 , srv39413807:84 , srv403699611:84 , srv432204217:84 , srv432777796:84 , srv441410298:84 , srv457460314:84 , srv470028780:84 , srv512217200:84 , srv586332404:84 , srv599190137:84 , srv60096818:84 , srv620228540:84 , srv635740612:84 , srv645193822:84 , srv652530187:84 , srv653349079:84 , srv725738440:84 , srv727887941:84 , srv732558905:84 , srv750540436:84 , srv787559780:84 , srv819554675:84 , srv854462171:84 , srv887464687:84 , srv922772728:84 , srv93120102:84 , srv953272379:84 , srv963824517:84 , srv993828097:84 , srv1090029262:85 , srv1390842652:85 , srv1780243488:85 , srv2127690119:85 , srv253060225:85 , srv694277461:85 , srv808541429:85 , srv391471813:684 }
2023-11-08 22:59:01,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:59:01,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv963824517=98, srv351685070=65, srv1814046737=47, srv1616058416=32, srv1008014144=1, srv1023182584=5, srv620228540=79, srv1084492301=8, srv1461728394=23, srv1622610644=34, srv332878008=62, srv2121175027=55, srv922772728=94, srv432777796=71, srv1697625864=38, srv1225873370=16, srv2029403673=52, srv326418616=61, srv1180178134=15, srv1713932214=39, srv653349079=83, srv953272379=96, srv1673197585=37, srv645193822=81, srv635740612=80, srv1476598138=24, srv162095538=33, srv1420911307=21, srv253060225=59, srv1458017686=22, srv1924984302=49, srv1107910630=12, srv284247200=60, srv694277461=84, srv1780243488=44, srv457460314=73, srv750540436=88, srv1292313859=17, srv1537980075=29, srv1584263858=31, srv165983488=36, srv100497238=0, srv599190137=77, srv1043258802=6, srv334108494=63, srv351337883=64, srv787559780=89, srv1176407167=14, srv1553207487=30, srv512217200=75, srv652530187=82, srv2096148912=54, srv218119069=58, srv819554675=91, srv2127690119=57, srv808541429=90, srv93120102=95, srv1480310794=25, srv1835866694=48, srv153060544=28, srv1090029262=9, srv1104343064=11, srv887464687=93, srv1009259880=3, srv384740078=66, srv1009048316=2, srv1311672615=18, srv17851264=45, srv39413807=68, srv1757880196=43, srv586332404=76, srv2052363143=53, srv2122699823=56, srv1750800702=40, srv441410298=72, srv105267111=7, srv1757703812=42, srv1750882245=41, srv1398726863=20, srv432204217=70, srv732558905=87, srv1797762391=46, srv470028780=74, srv955537589=97, srv727887941=86, srv60096818=78, srv1390842652=19, srv993828097=99, srv117108809=13, srv725738440=85, srv1641245599=35, srv391471813=67, srv2005477710=50, srv1092111886=10, srv1018496621=4, srv1521042310=27, srv1509560974=26, srv2017690895=51, srv854462171=92, srv403699611=69} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:01,785 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-08 22:59:01,786 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-08 22:59:01,787 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-08 22:59:01,788 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-08 22:59:01,789 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-08 22:59:01,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-08 22:59:01,790 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-08 22:59:01,880 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.397985672269699E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014213381090374027); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,880 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1018496621:83 , srv1092111886:83 , srv1104343064:83 , srv1553207487:83 , srv1713932214:83 , srv384740078:83 , srv955537589:83 , srv100497238:84 , srv1008014144:84 , srv1009048316:84 , srv1009259880:84 , srv1023182584:84 , srv1043258802:84 , srv105267111:84 , srv1084492301:84 , srv1107910630:84 , srv117108809:84 , srv1176407167:84 , srv1180178134:84 , srv1225873370:84 , srv1292313859:84 , srv1311672615:84 , srv1398726863:84 , srv1420911307:84 , srv1458017686:84 , srv1461728394:84 , srv1476598138:84 , srv1480310794:84 , srv1509560974:84 , srv1521042310:84 , srv153060544:84 , srv1537980075:84 , srv1584263858:84 , srv1616058416:84 , srv162095538:84 , srv1622610644:84 , srv1641245599:84 , srv165983488:84 , srv1673197585:84 , srv1697625864:84 , srv1750800702:84 , srv1750882245:84 , srv1757703812:84 , srv1757880196:84 , srv17851264:84 , srv1797762391:84 , srv1814046737:84 , srv1835866694:84 , srv1924984302:84 , srv2005477710:84 , srv2017690895:84 , srv2029403673:84 , srv2052363143:84 , srv2096148912:84 , srv2121175027:84 , srv2122699823:84 , srv218119069:84 , srv284247200:84 , srv326418616:84 , srv332878008:84 , srv334108494:84 , srv351337883:84 , srv351685070:84 , srv39413807:84 , srv403699611:84 , srv432204217:84 , srv432777796:84 , srv441410298:84 , srv457460314:84 , srv470028780:84 , srv512217200:84 , srv586332404:84 , srv599190137:84 , srv60096818:84 , srv620228540:84 , srv635740612:84 , srv645193822:84 , srv652530187:84 , srv653349079:84 , srv725738440:84 , srv727887941:84 , srv732558905:84 , srv750540436:84 , srv787559780:84 , srv819554675:84 , srv854462171:84 , srv887464687:84 , srv922772728:84 , srv93120102:84 , srv953272379:84 , srv963824517:84 , srv993828097:84 , srv1090029262:85 , srv1390842652:85 , srv1780243488:85 , srv2127690119:85 , srv253060225:85 , srv694277461:85 , srv808541429:85 , srv391471813:684 }
2023-11-08 22:59:01,906 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=369 (was 375), ProcessCount=166 (was 166), AvailableMemoryMB=6500 (was 6978)
2023-11-08 22:59:01,912 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=166, AvailableMemoryMB=6500
2023-11-08 22:59:01,913 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-08 22:59:01,913 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-08 22:59:01,921 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1035837945:84 , srv1060530771:84 , srv1112044674:84 , srv1165821154:84 , srv1184067483:84 , srv1219847546:84 , srv1336492179:84 , srv1376118084:84 , srv1393413581:84 , srv1419342923:84 , srv1473451167:84 , srv1517792144:84 , srv1549020634:84 , srv156097349:84 , srv1575390634:84 , srv173824213:84 , srv1825153901:84 , srv1859846891:84 , srv187224608:84 , srv2010896891:84 , srv2055068547:84 , srv2128834627:84 , srv308009621:84 , srv39953517:84 , srv677910716:84 , srv760944804:84 , srv907438761:84 , srv933680981:84 , srv946300498:84 , srv7352376:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-08 22:59:01,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 22:59:01,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv933680981=28, srv1825153901=16, srv1419342923=9, srv2055068547=20, srv1060530771=1, srv156097349=13, srv1473451167=10, srv677910716=24, srv1393413581=8, srv1549020634=12, srv1035837945=0, srv1165821154=3, srv1336492179=6, srv1859846891=17, srv308009621=22, srv187224608=18, srv2128834627=21, srv7352376=25, srv1575390634=14, srv1376118084=7, srv1112044674=2, srv1517792144=11, srv39953517=23, srv760944804=26, srv1219847546=5, srv907438761=27, srv173824213=15, srv946300498=29, srv2010896891=19, srv1184067483=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 22:59:01,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 22:59:01,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-08 22:59:01,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.073898203481876, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 22:59:01,947 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912425894090454, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.073898203481876, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-08 23:01:01,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9289652 different iterations.  Found a solution that moves 1837 regions; Going from a computed imbalance of 0.9912425894090454 to a new imbalance of 4.2876496417733574E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6803703703703704, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 23:01:01,943 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035837945:90 , srv1060530771:90 , srv1112044674:90 , srv1165821154:90 , srv1184067483:90 , srv1219847546:90 , srv1336492179:90 , srv1376118084:90 , srv1393413581:90 , srv1419342923:90 , srv1473451167:90 , srv1517792144:90 , srv1549020634:90 , srv156097349:90 , srv1575390634:90 , srv173824213:90 , srv1825153901:90 , srv1859846891:90 , srv187224608:90 , srv2010896891:90 , srv2055068547:90 , srv2128834627:90 , srv308009621:90 , srv39953517:90 , srv677910716:90 , srv7352376:90 , srv760944804:90 , srv907438761:90 , srv933680981:90 , srv946300498:90 }
2023-11-08 23:01:01,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-08 23:01:01,944 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv933680981=28, srv1825153901=16, srv1419342923=9, srv2055068547=20, srv1060530771=1, srv156097349=13, srv1473451167=10, srv677910716=24, srv1393413581=8, srv1549020634=12, srv1035837945=0, srv1165821154=3, srv1336492179=6, srv1859846891=17, srv308009621=22, srv187224608=18, srv2128834627=21, srv7352376=25, srv1575390634=14, srv1376118084=7, srv1112044674=2, srv1517792144=11, srv39953517=23, srv760944804=26, srv1219847546=5, srv907438761=27, srv173824213=15, srv946300498=29, srv2010896891=19, srv1184067483=4} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-08 23:01:01,947 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-08 23:01:01,948 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-08 23:01:01,949 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-08 23:01:01,962 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-08 23:01:01,963 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035837945:90 , srv1060530771:90 , srv1112044674:90 , srv1165821154:90 , srv1184067483:90 , srv1219847546:90 , srv1336492179:90 , srv1376118084:90 , srv1393413581:90 , srv1419342923:90 , srv1473451167:90 , srv1517792144:90 , srv1549020634:90 , srv156097349:90 , srv1575390634:90 , srv173824213:90 , srv1825153901:90 , srv1859846891:90 , srv187224608:90 , srv2010896891:90 , srv2055068547:90 , srv2128834627:90 , srv308009621:90 , srv39953517:90 , srv677910716:90 , srv7352376:90 , srv760944804:90 , srv907438761:90 , srv933680981:90 , srv946300498:90 }
2023-11-08 23:01:01,973 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=170 (was 369), ProcessCount=166 (was 166), AvailableMemoryMB=6462 (was 6500)