2023-10-22 00:53:26,938 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-10-22 00:53:27,236 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-10-22 00:53:27,349 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-10-22 00:53:27,407 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-22 00:53:27,414 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-22 00:53:27,427 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=483, ProcessCount=169, AvailableMemoryMB=8603
2023-10-22 00:53:27,428 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-22 00:53:27,429 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-10-22 00:53:27,549 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1011709643:84 , srv1026983295:84 , srv1035481656:84 , srv1052652268:84 , srv1081025354:84 , srv1104596904:84 , srv1120081197:84 , srv1129179730:84 , srv1129389122:84 , srv1133046195:84 , srv1208509723:84 , srv1229342074:84 , srv1250833360:84 , srv1256508145:84 , srv1260945836:84 , srv131333184:84 , srv1340083962:84 , srv1342110933:84 , srv1366770654:84 , srv1413927528:84 , srv1443859885:84 , srv1463923883:84 , srv1497967:84 , srv1508008575:84 , srv1586143672:84 , srv1593034479:84 , srv164959541:84 , srv169399239:84 , srv1755464080:84 , srv175556591:84 , srv1756224875:84 , srv1757664448:84 , srv1825027008:84 , srv1834483453:84 , srv1839828777:84 , srv188181953:84 , srv1929068588:84 , srv1942541932:84 , srv1986705052:84 , srv2009932633:84 , srv2015223158:84 , srv2016304923:84 , srv2020724925:84 , srv2028741483:84 , srv2048095139:84 , srv2073814577:84 , srv2079441234:84 , srv2118682320:84 , srv212515641:84 , srv224717055:84 , srv238775373:84 , srv267702225:84 , srv276802368:84 , srv285801520:84 , srv301170245:84 , srv315896751:84 , srv328209053:84 , srv328531671:84 , srv329296037:84 , srv385793727:84 , srv405088442:84 , srv43131121:84 , srv431347550:84 , srv437720230:84 , srv454907181:84 , srv478883186:84 , srv498133412:84 , srv532721374:84 , srv534029994:84 , srv552246464:84 , srv560445021:84 , srv567066982:84 , srv601462458:84 , srv603384974:84 , srv61250164:84 , srv612900598:84 , srv636777669:84 , srv652283125:84 , srv662332049:84 , srv664545136:84 , srv667030501:84 , srv674698364:84 , srv69887766:84 , srv709423068:84 , srv736596184:84 , srv748486044:84 , srv759432081:84 , srv76722340:84 , srv769599748:84 , srv783249575:84 , srv805746097:84 , srv813103257:84 , srv822552614:84 , srv92372341:84 , srv932525089:84 , srv965318465:84 , srv968120788:84 , srv988720588:84 , srv999194879:84 , srv582010187:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-10-22 00:53:27,557 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:27,563 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv965318465=96, srv1366770654=18, srv612900598=76, srv498133412=66, srv674698364=82, srv92372341=94, srv1081025354=4, srv1011709643=0, srv1508008575=23, srv2015223158=40, srv164959541=26, srv999194879=99, srv662332049=79, srv805746097=91, srv1757664448=31, srv1104596904=5, srv175556591=29, srv532721374=67, srv968120788=97, srv131333184=15, srv2028741483=43, srv601462458=73, srv1250833360=12, srv2009932633=39, srv2048095139=44, srv582010187=72, srv988720588=98, srv636777669=77, srv1413927528=19, srv61250164=75, srv652283125=78, srv1129389122=8, srv329296037=58, srv69887766=83, srv813103257=92, srv276802368=52, srv315896751=55, srv1340083962=16, srv1052652268=3, srv2079441234=46, srv2020724925=42, srv405088442=60, srv709423068=84, srv431347550=62, srv188181953=35, srv238775373=50, srv783249575=90, srv1342110933=17, srv1497967=22, srv267702225=51, srv736596184=85, srv1756224875=30, srv1120081197=6, srv328209053=56, srv285801520=53, srv1256508145=13, srv301170245=54, srv454907181=64, srv43131121=61, srv1839828777=34, srv552246464=69, srv224717055=49, srv1834483453=33, srv822552614=93, srv1026983295=1, srv1593034479=25, srv1986705052=38, srv567066982=71, srv2016304923=41, srv1260945836=14, srv1463923883=21, srv2118682320=47, srv1942541932=37, srv478883186=65, srv385793727=59, srv1208509723=10, srv1755464080=28, srv2073814577=45, srv759432081=87, srv769599748=89, srv1929068588=36, srv603384974=74, srv437720230=63, srv1825027008=32, srv748486044=86, srv667030501=81, srv1229342074=11, srv212515641=48, srv169399239=27, srv664545136=80, srv932525089=95, srv1133046195=9, srv328531671=57, srv1035481656=2, srv534029994=68, srv76722340=88, srv560445021=70, srv1443859885=20, srv1129179730=7, srv1586143672=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:27,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:27,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:27,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:27,635 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:27,636 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:27,637 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:27,638 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-22 00:53:27,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-22 00:53:27,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-22 00:53:27,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-22 00:53:27,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-22 00:53:27,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-22 00:53:27,639 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-22 00:53:27,640 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-22 00:53:27,641 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-22 00:53:27,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-22 00:53:27,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-22 00:53:27,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-22 00:53:27,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-22 00:53:27,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-22 00:53:27,642 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-22 00:53:27,643 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-22 00:53:27,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-22 00:53:27,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-22 00:53:27,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-22 00:53:27,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-22 00:53:27,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-22 00:53:27,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-22 00:53:27,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-22 00:53:27,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-22 00:53:27,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-22 00:53:27,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-22 00:53:27,880 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:27,881 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-22 00:53:27,881 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-22 00:53:28,370 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 693 ms to try 5000 different iterations.  Found a solution that moves 4665 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18396620757915677. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0384858058448517, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05039239961932691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5183333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0384858058448517, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17933333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:28,396 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2118682320:81 , srv1366770654:82 , srv1839828777:82 , srv238775373:82 , srv285801520:82 , srv652283125:82 , srv1035481656:83 , srv1413927528:83 , srv1593034479:83 , srv224717055:83 , srv267702225:83 , srv636777669:83 , srv769599748:83 , srv92372341:83 , srv1026983295:84 , srv1052652268:84 , srv1757664448:84 , srv2073814577:84 , srv437720230:84 , srv560445021:84 , srv1011709643:85 , srv1129389122:85 , srv1208509723:85 , srv1260945836:85 , srv131333184:85 , srv1342110933:85 , srv1497967:85 , srv169399239:85 , srv2028741483:85 , srv498133412:85 , srv567066982:85 , srv736596184:85 , srv1463923883:86 , srv1586143672:86 , srv164959541:86 , srv1755464080:86 , srv175556591:86 , srv1756224875:86 , srv188181953:86 , srv1986705052:86 , srv328531671:86 , srv405088442:86 , srv532721374:86 , srv601462458:86 , srv603384974:86 , srv61250164:86 , srv662332049:86 , srv76722340:86 , srv783249575:86 , srv932525089:86 , srv1129179730:87 , srv1256508145:87 , srv1942541932:87 , srv2020724925:87 , srv2048095139:87 , srv2079441234:87 , srv43131121:87 , srv478883186:87 , srv552246464:87 , srv612900598:87 , srv664545136:87 , srv748486044:87 , srv822552614:87 , srv1081025354:88 , srv1120081197:88 , srv1250833360:88 , srv1508008575:88 , srv1834483453:88 , srv301170245:88 , srv328209053:88 , srv329296037:88 , srv667030501:88 , srv674698364:88 , srv69887766:88 , srv709423068:88 , srv805746097:88 , srv968120788:88 , srv999194879:88 , srv1133046195:89 , srv1229342074:89 , srv1340083962:89 , srv1443859885:89 , srv1929068588:89 , srv2009932633:89 , srv212515641:89 , srv276802368:89 , srv454907181:89 , srv813103257:89 , srv988720588:89 , srv315896751:90 , srv431347550:90 , srv534029994:90 , srv965318465:90 , srv1825027008:91 , srv2015223158:91 , srv385793727:91 , srv759432081:92 , srv1104596904:93 , srv2016304923:93 , srv582010187:432 }
2023-10-22 00:53:28,403 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:28,404 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv965318465=96, srv1366770654=18, srv612900598=76, srv498133412=66, srv674698364=82, srv92372341=94, srv1081025354=4, srv1011709643=0, srv1508008575=23, srv2015223158=40, srv164959541=26, srv999194879=99, srv662332049=79, srv805746097=91, srv1757664448=31, srv1104596904=5, srv175556591=29, srv532721374=67, srv968120788=97, srv131333184=15, srv2028741483=43, srv601462458=73, srv1250833360=12, srv2009932633=39, srv2048095139=44, srv582010187=72, srv988720588=98, srv636777669=77, srv1413927528=19, srv61250164=75, srv652283125=78, srv1129389122=8, srv329296037=58, srv69887766=83, srv813103257=92, srv276802368=52, srv315896751=55, srv1340083962=16, srv1052652268=3, srv2079441234=46, srv2020724925=42, srv405088442=60, srv709423068=84, srv431347550=62, srv188181953=35, srv238775373=50, srv783249575=90, srv1342110933=17, srv1497967=22, srv267702225=51, srv736596184=85, srv1756224875=30, srv1120081197=6, srv328209053=56, srv285801520=53, srv1256508145=13, srv301170245=54, srv454907181=64, srv43131121=61, srv1839828777=34, srv552246464=69, srv224717055=49, srv1834483453=33, srv822552614=93, srv1026983295=1, srv1593034479=25, srv1986705052=38, srv567066982=71, srv2016304923=41, srv1260945836=14, srv1463923883=21, srv2118682320=47, srv1942541932=37, srv478883186=65, srv385793727=59, srv1208509723=10, srv1755464080=28, srv2073814577=45, srv759432081=87, srv769599748=89, srv1929068588=36, srv603384974=74, srv437720230=63, srv1825027008=32, srv748486044=86, srv667030501=81, srv1229342074=11, srv212515641=48, srv169399239=27, srv664545136=80, srv932525089=95, srv1133046195=9, srv328531671=57, srv1035481656=2, srv534029994=68, srv76722340=88, srv560445021=70, srv1443859885=20, srv1129179730=7, srv1586143672=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:28,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:28,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:28,429 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:28,430 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:28,431 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:28,432 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-22 00:53:28,433 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-22 00:53:28,434 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-22 00:53:28,435 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-22 00:53:28,436 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-22 00:53:28,437 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-22 00:53:28,438 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-22 00:53:28,439 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-22 00:53:28,440 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-22 00:53:28,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-22 00:53:28,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-22 00:53:28,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-22 00:53:28,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-22 00:53:28,606 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0384858058448517, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05039239961932691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0384858058448517, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17933333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:28,606 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-22 00:53:28,606 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18393354255099312, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0384858058448517, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05039239961932691, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0384858058448517, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17933333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24516666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-22 00:53:28,891 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 428 ms to try 5000 different iterations.  Found a solution that moves 2633 regions; Going from a computed imbalance of 0.18393354255099312 to a new imbalance of 0.014965681035410882. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019712359810988383); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03283123529846759, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29255555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019712359810988383); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:28,902 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1104596904:80 , srv965318465:80 , srv385793727:81 , srv534029994:81 , srv1229342074:82 , srv1825027008:82 , srv1929068588:82 , srv2015223158:82 , srv276802368:82 , srv69887766:82 , srv759432081:82 , srv968120788:82 , srv301170245:83 , srv315896751:83 , srv552246464:83 , srv674698364:83 , srv709423068:83 , srv813103257:83 , srv988720588:83 , srv1129179730:84 , srv1256508145:84 , srv164959541:84 , srv1834483453:84 , srv1942541932:84 , srv2009932633:84 , srv2020724925:84 , srv2048095139:84 , srv2079441234:84 , srv431347550:84 , srv805746097:84 , srv999194879:84 , srv1011709643:85 , srv1340083962:85 , srv1443859885:85 , srv1463923883:85 , srv1508008575:85 , srv1586143672:85 , srv169399239:85 , srv1986705052:85 , srv2016304923:85 , srv2028741483:85 , srv212515641:85 , srv328209053:85 , srv329296037:85 , srv405088442:85 , srv454907181:85 , srv478883186:85 , srv601462458:85 , srv612900598:85 , srv76722340:85 , srv932525089:85 , srv1120081197:86 , srv1129389122:86 , srv1133046195:86 , srv1250833360:86 , srv1497967:86 , srv188181953:86 , srv43131121:86 , srv532721374:86 , srv664545136:86 , srv667030501:86 , srv783249575:86 , srv1081025354:87 , srv1208509723:87 , srv1260945836:87 , srv1342110933:87 , srv1755464080:87 , srv175556591:87 , srv1756224875:87 , srv328531671:87 , srv603384974:87 , srv61250164:87 , srv736596184:87 , srv748486044:87 , srv769599748:87 , srv2073814577:88 , srv267702225:88 , srv437720230:88 , srv498133412:88 , srv560445021:88 , srv567066982:88 , srv652283125:88 , srv92372341:88 , srv1035481656:89 , srv1757664448:89 , srv1839828777:89 , srv224717055:89 , srv238775373:89 , srv285801520:89 , srv636777669:89 , srv662332049:89 , srv822552614:89 , srv1026983295:90 , srv1052652268:90 , srv131333184:90 , srv1366770654:91 , srv1413927528:91 , srv1593034479:91 , srv2118682320:91 , srv582010187:517 }
2023-10-22 00:53:28,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:28,910 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv965318465=96, srv1366770654=18, srv612900598=76, srv498133412=66, srv674698364=82, srv92372341=94, srv1081025354=4, srv1011709643=0, srv1508008575=23, srv2015223158=40, srv164959541=26, srv999194879=99, srv662332049=79, srv805746097=91, srv1757664448=31, srv1104596904=5, srv175556591=29, srv532721374=67, srv968120788=97, srv131333184=15, srv2028741483=43, srv601462458=73, srv1250833360=12, srv2009932633=39, srv2048095139=44, srv582010187=72, srv988720588=98, srv636777669=77, srv1413927528=19, srv61250164=75, srv652283125=78, srv1129389122=8, srv329296037=58, srv69887766=83, srv813103257=92, srv276802368=52, srv315896751=55, srv1340083962=16, srv1052652268=3, srv2079441234=46, srv2020724925=42, srv405088442=60, srv709423068=84, srv431347550=62, srv188181953=35, srv238775373=50, srv783249575=90, srv1342110933=17, srv1497967=22, srv267702225=51, srv736596184=85, srv1756224875=30, srv1120081197=6, srv328209053=56, srv285801520=53, srv1256508145=13, srv301170245=54, srv454907181=64, srv43131121=61, srv1839828777=34, srv552246464=69, srv224717055=49, srv1834483453=33, srv822552614=93, srv1026983295=1, srv1593034479=25, srv1986705052=38, srv567066982=71, srv2016304923=41, srv1260945836=14, srv1463923883=21, srv2118682320=47, srv1942541932=37, srv478883186=65, srv385793727=59, srv1208509723=10, srv1755464080=28, srv2073814577=45, srv759432081=87, srv769599748=89, srv1929068588=36, srv603384974=74, srv437720230=63, srv1825027008=32, srv748486044=86, srv667030501=81, srv1229342074=11, srv212515641=48, srv169399239=27, srv664545136=80, srv932525089=95, srv1133046195=9, srv328531671=57, srv1035481656=2, srv534029994=68, srv76722340=88, srv560445021=70, srv1443859885=20, srv1129179730=7, srv1586143672=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:28,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:28,926 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-22 00:53:28,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-22 00:53:28,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-22 00:53:28,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-22 00:53:28,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-22 00:53:28,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-22 00:53:28,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-22 00:53:28,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-22 00:53:28,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-22 00:53:29,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019712359810988383); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03283123529846759, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019712359810988383); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:29,094 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-22 00:53:29,094 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014947244375356246, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.019712359810988383); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03283123529846759, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.019712359810988383); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01175); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04583333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-22 00:53:29,371 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 399 ms to try 5000 different iterations.  Found a solution that moves 962 regions; Going from a computed imbalance of 0.014947244375356246 to a new imbalance of 7.476952626352765E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0056567660698823964); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008554309236258637); MoveCostFunction : (multiplier=7.0, imbalance=0.1068888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0056567660698823964); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:29,372 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2016304923:79 , srv1133046195:82 , srv822552614:82 , srv1120081197:83 , srv1250833360:83 , srv131333184:83 , srv1340083962:83 , srv1443859885:83 , srv212515641:83 , srv667030501:83 , srv1026983295:84 , srv1052652268:84 , srv1104596904:84 , srv1366770654:84 , srv1413927528:84 , srv1508008575:84 , srv1593034479:84 , srv1825027008:84 , srv2015223158:84 , srv315896751:84 , srv328209053:84 , srv431347550:84 , srv454907181:84 , srv498133412:84 , srv603384974:84 , srv61250164:84 , srv664545136:84 , srv748486044:84 , srv759432081:84 , srv1035481656:85 , srv1081025354:85 , srv1208509723:85 , srv1260945836:85 , srv1342110933:85 , srv1755464080:85 , srv175556591:85 , srv1756224875:85 , srv1757664448:85 , srv1834483453:85 , srv188181953:85 , srv2009932633:85 , srv2073814577:85 , srv328531671:85 , srv329296037:85 , srv43131121:85 , srv478883186:85 , srv560445021:85 , srv567066982:85 , srv612900598:85 , srv662332049:85 , srv813103257:85 , srv932525089:85 , srv988720588:85 , srv1129179730:86 , srv1129389122:86 , srv1586143672:86 , srv1839828777:86 , srv1929068588:86 , srv1986705052:86 , srv2118682320:86 , srv224717055:86 , srv238775373:86 , srv276802368:86 , srv285801520:86 , srv301170245:86 , srv385793727:86 , srv405088442:86 , srv437720230:86 , srv532721374:86 , srv534029994:86 , srv636777669:86 , srv674698364:86 , srv709423068:86 , srv736596184:86 , srv76722340:86 , srv805746097:86 , srv92372341:86 , srv999194879:86 , srv1011709643:87 , srv1229342074:87 , srv1256508145:87 , srv1463923883:87 , srv1497967:87 , srv164959541:87 , srv169399239:87 , srv1942541932:87 , srv2020724925:87 , srv2048095139:87 , srv2079441234:87 , srv267702225:87 , srv601462458:87 , srv652283125:87 , srv69887766:87 , srv769599748:87 , srv783249575:87 , srv2028741483:88 , srv552246464:88 , srv965318465:88 , srv968120788:88 , srv582010187:559 }
2023-10-22 00:53:29,376 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:29,377 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv965318465=96, srv1366770654=18, srv612900598=76, srv498133412=66, srv674698364=82, srv92372341=94, srv1081025354=4, srv1011709643=0, srv1508008575=23, srv2015223158=40, srv164959541=26, srv999194879=99, srv662332049=79, srv805746097=91, srv1757664448=31, srv1104596904=5, srv175556591=29, srv532721374=67, srv968120788=97, srv131333184=15, srv2028741483=43, srv601462458=73, srv1250833360=12, srv2009932633=39, srv2048095139=44, srv582010187=72, srv988720588=98, srv636777669=77, srv1413927528=19, srv61250164=75, srv652283125=78, srv1129389122=8, srv329296037=58, srv69887766=83, srv813103257=92, srv276802368=52, srv315896751=55, srv1340083962=16, srv1052652268=3, srv2079441234=46, srv2020724925=42, srv405088442=60, srv709423068=84, srv431347550=62, srv188181953=35, srv238775373=50, srv783249575=90, srv1342110933=17, srv1497967=22, srv267702225=51, srv736596184=85, srv1756224875=30, srv1120081197=6, srv328209053=56, srv285801520=53, srv1256508145=13, srv301170245=54, srv454907181=64, srv43131121=61, srv1839828777=34, srv552246464=69, srv224717055=49, srv1834483453=33, srv822552614=93, srv1026983295=1, srv1593034479=25, srv1986705052=38, srv567066982=71, srv2016304923=41, srv1260945836=14, srv1463923883=21, srv2118682320=47, srv1942541932=37, srv478883186=65, srv385793727=59, srv1208509723=10, srv1755464080=28, srv2073814577=45, srv759432081=87, srv769599748=89, srv1929068588=36, srv603384974=74, srv437720230=63, srv1825027008=32, srv748486044=86, srv667030501=81, srv1229342074=11, srv212515641=48, srv169399239=27, srv664545136=80, srv932525089=95, srv1133046195=9, srv328531671=57, srv1035481656=2, srv534029994=68, srv76722340=88, srv560445021=70, srv1443859885=20, srv1129179730=7, srv1586143672=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:29,388 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:29,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-22 00:53:29,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-22 00:53:29,391 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-22 00:53:29,392 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-22 00:53:29,393 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-22 00:53:29,394 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-22 00:53:29,395 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-22 00:53:29,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0056567660698823964); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008554309236258637); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0056567660698823964); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:29,516 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-22 00:53:29,516 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.40959194662409E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0056567660698823964); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008554309236258637); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0056567660698823964); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=3.333333333333333E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004166666666666667); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-22 00:53:29,887 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 483 ms to try 5000 different iterations.  Found a solution that moves 233 regions; Going from a computed imbalance of 7.40959194662409E-4 to a new imbalance of 2.7130161306924523E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.02588888888888889, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:29,888 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1081025354:82 , srv662332049:82 , srv783249575:83 , srv1011709643:84 , srv1026983295:84 , srv1052652268:84 , srv1129389122:84 , srv1229342074:84 , srv1256508145:84 , srv131333184:84 , srv1413927528:84 , srv1497967:84 , srv1586143672:84 , srv1593034479:84 , srv1755464080:84 , srv175556591:84 , srv1756224875:84 , srv1757664448:84 , srv1942541932:84 , srv2009932633:84 , srv2020724925:84 , srv2028741483:84 , srv2048095139:84 , srv2079441234:84 , srv2118682320:84 , srv224717055:84 , srv267702225:84 , srv328209053:84 , srv328531671:84 , srv329296037:84 , srv385793727:84 , srv405088442:84 , srv43131121:84 , srv431347550:84 , srv437720230:84 , srv454907181:84 , srv532721374:84 , srv552246464:84 , srv567066982:84 , srv601462458:84 , srv636777669:84 , srv736596184:84 , srv759432081:84 , srv805746097:84 , srv822552614:84 , srv965318465:84 , srv968120788:84 , srv999194879:84 , srv1035481656:85 , srv1104596904:85 , srv1120081197:85 , srv1129179730:85 , srv1133046195:85 , srv1208509723:85 , srv1250833360:85 , srv1340083962:85 , srv1342110933:85 , srv1366770654:85 , srv1443859885:85 , srv1463923883:85 , srv1508008575:85 , srv164959541:85 , srv169399239:85 , srv1825027008:85 , srv1834483453:85 , srv1839828777:85 , srv188181953:85 , srv1929068588:85 , srv1986705052:85 , srv2015223158:85 , srv2073814577:85 , srv212515641:85 , srv238775373:85 , srv276802368:85 , srv285801520:85 , srv301170245:85 , srv315896751:85 , srv478883186:85 , srv498133412:85 , srv534029994:85 , srv560445021:85 , srv603384974:85 , srv61250164:85 , srv612900598:85 , srv652283125:85 , srv664545136:85 , srv667030501:85 , srv674698364:85 , srv69887766:85 , srv709423068:85 , srv748486044:85 , srv76722340:85 , srv769599748:85 , srv813103257:85 , srv92372341:85 , srv988720588:85 , srv1260945836:86 , srv2016304923:86 , srv932525089:86 , srv582010187:635 }
2023-10-22 00:53:29,891 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:29,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv965318465=96, srv1366770654=18, srv612900598=76, srv498133412=66, srv674698364=82, srv92372341=94, srv1081025354=4, srv1011709643=0, srv1508008575=23, srv2015223158=40, srv164959541=26, srv999194879=99, srv662332049=79, srv805746097=91, srv1757664448=31, srv1104596904=5, srv175556591=29, srv532721374=67, srv968120788=97, srv131333184=15, srv2028741483=43, srv601462458=73, srv1250833360=12, srv2009932633=39, srv2048095139=44, srv582010187=72, srv988720588=98, srv636777669=77, srv1413927528=19, srv61250164=75, srv652283125=78, srv1129389122=8, srv329296037=58, srv69887766=83, srv813103257=92, srv276802368=52, srv315896751=55, srv1340083962=16, srv1052652268=3, srv2079441234=46, srv2020724925=42, srv405088442=60, srv709423068=84, srv431347550=62, srv188181953=35, srv238775373=50, srv783249575=90, srv1342110933=17, srv1497967=22, srv267702225=51, srv736596184=85, srv1756224875=30, srv1120081197=6, srv328209053=56, srv285801520=53, srv1256508145=13, srv301170245=54, srv454907181=64, srv43131121=61, srv1839828777=34, srv552246464=69, srv224717055=49, srv1834483453=33, srv822552614=93, srv1026983295=1, srv1593034479=25, srv1986705052=38, srv567066982=71, srv2016304923=41, srv1260945836=14, srv1463923883=21, srv2118682320=47, srv1942541932=37, srv478883186=65, srv385793727=59, srv1208509723=10, srv1755464080=28, srv2073814577=45, srv759432081=87, srv769599748=89, srv1929068588=36, srv603384974=74, srv437720230=63, srv1825027008=32, srv748486044=86, srv667030501=81, srv1229342074=11, srv212515641=48, srv169399239=27, srv664545136=80, srv932525089=95, srv1133046195=9, srv328531671=57, srv1035481656=2, srv534029994=68, srv76722340=88, srv560445021=70, srv1443859885=20, srv1129179730=7, srv1586143672=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:29,902 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:29,903 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-22 00:53:29,904 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-22 00:53:29,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-22 00:53:29,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-22 00:53:29,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-22 00:53:29,908 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-22 00:53:30,014 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:30,014 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-10-22 00:53:30,015 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.5498660436157198E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=3.5313429231318235E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0019534423844256277); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=3.5313429231318235E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-10-22 00:53:30,274 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 356 ms to try 5000 different iterations.  Found a solution that moves 21 regions; Going from a computed imbalance of 2.5498660436157198E-5 to a new imbalance of 7.220284439188305E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0023333333333333335); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:30,274 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1260945836:83 , srv1463923883:83 , srv2016304923:83 , srv748486044:83 , srv1026983295:84 , srv1035481656:84 , srv1052652268:84 , srv1081025354:84 , srv1104596904:84 , srv1120081197:84 , srv1129179730:84 , srv1133046195:84 , srv1208509723:84 , srv1229342074:84 , srv1250833360:84 , srv1256508145:84 , srv131333184:84 , srv1340083962:84 , srv1342110933:84 , srv1366770654:84 , srv1413927528:84 , srv1443859885:84 , srv1497967:84 , srv1508008575:84 , srv1593034479:84 , srv164959541:84 , srv169399239:84 , srv1755464080:84 , srv175556591:84 , srv1756224875:84 , srv1757664448:84 , srv1825027008:84 , srv1834483453:84 , srv1839828777:84 , srv188181953:84 , srv1929068588:84 , srv1942541932:84 , srv1986705052:84 , srv2009932633:84 , srv2015223158:84 , srv2020724925:84 , srv2028741483:84 , srv2048095139:84 , srv2073814577:84 , srv2079441234:84 , srv2118682320:84 , srv212515641:84 , srv224717055:84 , srv238775373:84 , srv267702225:84 , srv276802368:84 , srv285801520:84 , srv301170245:84 , srv315896751:84 , srv328531671:84 , srv329296037:84 , srv385793727:84 , srv43131121:84 , srv431347550:84 , srv437720230:84 , srv454907181:84 , srv478883186:84 , srv498133412:84 , srv532721374:84 , srv534029994:84 , srv552246464:84 , srv560445021:84 , srv567066982:84 , srv601462458:84 , srv603384974:84 , srv61250164:84 , srv612900598:84 , srv636777669:84 , srv652283125:84 , srv662332049:84 , srv664545136:84 , srv667030501:84 , srv674698364:84 , srv69887766:84 , srv709423068:84 , srv736596184:84 , srv759432081:84 , srv76722340:84 , srv769599748:84 , srv783249575:84 , srv805746097:84 , srv813103257:84 , srv822552614:84 , srv92372341:84 , srv932525089:84 , srv965318465:84 , srv968120788:84 , srv988720588:84 , srv999194879:84 , srv1011709643:85 , srv1129389122:85 , srv1586143672:85 , srv328209053:85 , srv405088442:85 , srv582010187:683 }
2023-10-22 00:53:30,278 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:30,279 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv965318465=96, srv1366770654=18, srv612900598=76, srv498133412=66, srv674698364=82, srv92372341=94, srv1081025354=4, srv1011709643=0, srv1508008575=23, srv2015223158=40, srv164959541=26, srv999194879=99, srv662332049=79, srv805746097=91, srv1757664448=31, srv1104596904=5, srv175556591=29, srv532721374=67, srv968120788=97, srv131333184=15, srv2028741483=43, srv601462458=73, srv1250833360=12, srv2009932633=39, srv2048095139=44, srv582010187=72, srv988720588=98, srv636777669=77, srv1413927528=19, srv61250164=75, srv652283125=78, srv1129389122=8, srv329296037=58, srv69887766=83, srv813103257=92, srv276802368=52, srv315896751=55, srv1340083962=16, srv1052652268=3, srv2079441234=46, srv2020724925=42, srv405088442=60, srv709423068=84, srv431347550=62, srv188181953=35, srv238775373=50, srv783249575=90, srv1342110933=17, srv1497967=22, srv267702225=51, srv736596184=85, srv1756224875=30, srv1120081197=6, srv328209053=56, srv285801520=53, srv1256508145=13, srv301170245=54, srv454907181=64, srv43131121=61, srv1839828777=34, srv552246464=69, srv224717055=49, srv1834483453=33, srv822552614=93, srv1026983295=1, srv1593034479=25, srv1986705052=38, srv567066982=71, srv2016304923=41, srv1260945836=14, srv1463923883=21, srv2118682320=47, srv1942541932=37, srv478883186=65, srv385793727=59, srv1208509723=10, srv1755464080=28, srv2073814577=45, srv759432081=87, srv769599748=89, srv1929068588=36, srv603384974=74, srv437720230=63, srv1825027008=32, srv748486044=86, srv667030501=81, srv1229342074=11, srv212515641=48, srv169399239=27, srv664545136=80, srv932525089=95, srv1133046195=9, srv328531671=57, srv1035481656=2, srv534029994=68, srv76722340=88, srv560445021=70, srv1443859885=20, srv1129179730=7, srv1586143672=24} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:30,302 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:30,303 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:30,304 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-10-22 00:53:30,305 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-10-22 00:53:30,306 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-10-22 00:53:30,307 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-10-22 00:53:30,308 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-10-22 00:53:30,309 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-10-22 00:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-10-22 00:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-10-22 00:53:30,310 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-10-22 00:53:30,420 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=7.073239296329447E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0015713484026367722); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:30,421 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1260945836:83 , srv1463923883:83 , srv2016304923:83 , srv748486044:83 , srv1026983295:84 , srv1035481656:84 , srv1052652268:84 , srv1081025354:84 , srv1104596904:84 , srv1120081197:84 , srv1129179730:84 , srv1133046195:84 , srv1208509723:84 , srv1229342074:84 , srv1250833360:84 , srv1256508145:84 , srv131333184:84 , srv1340083962:84 , srv1342110933:84 , srv1366770654:84 , srv1413927528:84 , srv1443859885:84 , srv1497967:84 , srv1508008575:84 , srv1593034479:84 , srv164959541:84 , srv169399239:84 , srv1755464080:84 , srv175556591:84 , srv1756224875:84 , srv1757664448:84 , srv1825027008:84 , srv1834483453:84 , srv1839828777:84 , srv188181953:84 , srv1929068588:84 , srv1942541932:84 , srv1986705052:84 , srv2009932633:84 , srv2015223158:84 , srv2020724925:84 , srv2028741483:84 , srv2048095139:84 , srv2073814577:84 , srv2079441234:84 , srv2118682320:84 , srv212515641:84 , srv224717055:84 , srv238775373:84 , srv267702225:84 , srv276802368:84 , srv285801520:84 , srv301170245:84 , srv315896751:84 , srv328531671:84 , srv329296037:84 , srv385793727:84 , srv43131121:84 , srv431347550:84 , srv437720230:84 , srv454907181:84 , srv478883186:84 , srv498133412:84 , srv532721374:84 , srv534029994:84 , srv552246464:84 , srv560445021:84 , srv567066982:84 , srv601462458:84 , srv603384974:84 , srv61250164:84 , srv612900598:84 , srv636777669:84 , srv652283125:84 , srv662332049:84 , srv664545136:84 , srv667030501:84 , srv674698364:84 , srv69887766:84 , srv709423068:84 , srv736596184:84 , srv759432081:84 , srv76722340:84 , srv769599748:84 , srv783249575:84 , srv805746097:84 , srv813103257:84 , srv822552614:84 , srv92372341:84 , srv932525089:84 , srv965318465:84 , srv968120788:84 , srv988720588:84 , srv999194879:84 , srv1011709643:85 , srv1129389122:85 , srv1586143672:85 , srv328209053:85 , srv405088442:85 , srv582010187:683 }
2023-10-22 00:53:30,449 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=483 (was 483), ProcessCount=169 (was 169), AvailableMemoryMB=8110 (was 8603)
2023-10-22 00:53:30,457 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=483, ProcessCount=169, AvailableMemoryMB=8110
2023-10-22 00:53:30,458 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-10-22 00:53:30,459 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-10-22 00:53:30,473 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1006507082:84 , srv1013837732:84 , srv1107676288:84 , srv1216112923:84 , srv129921266:84 , srv1336463186:84 , srv1360912572:84 , srv1416103594:84 , srv1530468159:84 , srv1555976460:84 , srv1556726537:84 , srv1659390251:84 , srv1680974939:84 , srv1685712163:84 , srv1731074707:84 , srv1777111618:84 , srv1886945667:84 , srv1933568291:84 , srv1988084711:84 , srv2128353588:84 , srv296907109:84 , srv327704768:84 , srv380417985:84 , srv472726269:84 , srv487791158:84 , srv612294073:84 , srv706994175:84 , srv858975082:84 , srv898255623:84 , srv1441290426:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-10-22 00:53:30,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:53:30,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416103594=7, srv1360912572=6, srv1556726537=11, srv1013837732=1, srv1659390251=12, srv1777111618=16, srv612294073=26, srv1731074707=15, srv327704768=22, srv1336463186=5, srv1441290426=8, srv1555976460=10, srv1933568291=18, srv706994175=27, srv1685712163=14, srv296907109=21, srv472726269=24, srv858975082=28, srv1107676288=2, srv1530468159=9, srv380417985=23, srv1988084711=19, srv898255623=29, srv129921266=4, srv1886945667=17, srv1680974939=13, srv487791158=25, srv2128353588=20, srv1006507082=0, srv1216112923=3} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:53:30,489 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:53:30,490 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:53:30,491 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-22 00:53:30,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0536536720951857, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:53:30,517 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.99123621042331, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0536536720951857, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-10-22 00:55:30,495 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5501946 different iterations.  Found a solution that moves 1856 regions; Going from a computed imbalance of 0.99123621042331 to a new imbalance of 4.3319965896196796E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6874074074074074, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:55:30,501 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1006507082:90 , srv1013837732:90 , srv1107676288:90 , srv1216112923:90 , srv129921266:90 , srv1336463186:90 , srv1360912572:90 , srv1416103594:90 , srv1441290426:90 , srv1530468159:90 , srv1555976460:90 , srv1556726537:90 , srv1659390251:90 , srv1680974939:90 , srv1685712163:90 , srv1731074707:90 , srv1777111618:90 , srv1886945667:90 , srv1933568291:90 , srv1988084711:90 , srv2128353588:90 , srv296907109:90 , srv327704768:90 , srv380417985:90 , srv472726269:90 , srv487791158:90 , srv612294073:90 , srv706994175:90 , srv858975082:90 , srv898255623:90 }
2023-10-22 00:55:30,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-10-22 00:55:30,502 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1416103594=7, srv1360912572=6, srv1556726537=11, srv1013837732=1, srv1659390251=12, srv1777111618=16, srv612294073=26, srv1731074707=15, srv327704768=22, srv1336463186=5, srv1441290426=8, srv1555976460=10, srv1933568291=18, srv706994175=27, srv1685712163=14, srv296907109=21, srv472726269=24, srv858975082=28, srv1107676288=2, srv1530468159=9, srv380417985=23, srv1988084711=19, srv898255623=29, srv129921266=4, srv1886945667=17, srv1680974939=13, srv487791158=25, srv2128353588=20, srv1006507082=0, srv1216112923=3} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-10-22 00:55:30,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-10-22 00:55:30,507 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-10-22 00:55:30,519 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-10-22 00:55:30,519 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1006507082:90 , srv1013837732:90 , srv1107676288:90 , srv1216112923:90 , srv129921266:90 , srv1336463186:90 , srv1360912572:90 , srv1416103594:90 , srv1441290426:90 , srv1530468159:90 , srv1555976460:90 , srv1556726537:90 , srv1659390251:90 , srv1680974939:90 , srv1685712163:90 , srv1731074707:90 , srv1777111618:90 , srv1886945667:90 , srv1933568291:90 , srv1988084711:90 , srv2128353588:90 , srv296907109:90 , srv327704768:90 , srv380417985:90 , srv472726269:90 , srv487791158:90 , srv612294073:90 , srv706994175:90 , srv858975082:90 , srv898255623:90 }
2023-10-22 00:55:30,532 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 483), ProcessCount=169 (was 169), AvailableMemoryMB=8046 (was 8110)