2023-04-28 08:55:41,978 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-04-28 08:55:42,397 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-04-28 08:55:42,544 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-04-28 08:55:42,629 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-28 08:55:42,639 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-28 08:55:42,657 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=396, ProcessCount=170, AvailableMemoryMB=6937
2023-04-28 08:55:42,660 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-28 08:55:42,660 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-04-28 08:55:42,827 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv103028325:84 , srv104424510:84 , srv104755312:84 , srv1066933731:84 , srv1096640647:84 , srv1113172680:84 , srv1115857553:84 , srv1120850972:84 , srv1148435655:84 , srv1159616674:84 , srv1166593989:84 , srv1189954606:84 , srv1191248077:84 , srv1216661652:84 , srv1217294750:84 , srv1303029047:84 , srv1310622926:84 , srv1312809948:84 , srv1313144862:84 , srv1339880015:84 , srv134036069:84 , srv1355811395:84 , srv1413854968:84 , srv1415646824:84 , srv1422898035:84 , srv1430192927:84 , srv1463760563:84 , srv1484537588:84 , srv1491216967:84 , srv1501772242:84 , srv1525570067:84 , srv1594578614:84 , srv1630387092:84 , srv1647133133:84 , srv1668828601:84 , srv1678600450:84 , srv168179345:84 , srv1694813821:84 , srv170514437:84 , srv1737214513:84 , srv1781412932:84 , srv1790482521:84 , srv1792104435:84 , srv1808050684:84 , srv1810451007:84 , srv1833703773:84 , srv1859425244:84 , srv1879764726:84 , srv1880949237:84 , srv1885241548:84 , srv1887621165:84 , srv1904065509:84 , srv1933466233:84 , srv1950731376:84 , srv2022677446:84 , srv2029315690:84 , srv2048469696:84 , srv2058563439:84 , srv2066111369:84 , srv2085400472:84 , srv2091509778:84 , srv2116194978:84 , srv227698348:84 , srv265376845:84 , srv273872085:84 , srv334862856:84 , srv337478680:84 , srv38135197:84 , srv392414637:84 , srv478158527:84 , srv485456238:84 , srv48990728:84 , srv496085330:84 , srv497238446:84 , srv550648444:84 , srv554170984:84 , srv559868756:84 , srv570989013:84 , srv575984120:84 , srv576488542:84 , srv576744135:84 , srv612949667:84 , srv63109996:84 , srv634834518:84 , srv63512975:84 , srv63735580:84 , srv645588265:84 , srv651289261:84 , srv692236010:84 , srv73946310:84 , srv740066283:84 , srv790864539:84 , srv797476008:84 , srv82750588:84 , srv86089871:84 , srv922652738:84 , srv937223888:84 , srv94254210:84 , srv986977854:84 , srv1599355320:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-04-28 08:55:42,843 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:42,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv82750588=94, srv2048469696=57, srv986977854=99, srv1525570067=30, srv227698348=63, srv1303029047=15, srv1115857553=6, srv392414637=69, srv1885241548=50, srv1790482521=42, srv1950731376=54, srv550648444=75, srv1312809948=17, srv103028325=0, srv1668828601=35, srv134036069=20, srv1694813821=38, srv1599355320=32, srv1880949237=49, srv1313144862=18, srv334862856=66, srv104755312=2, srv740066283=91, srv86089871=95, srv1216661652=13, srv1491216967=28, srv1413854968=22, srv63512975=85, srv1422898035=24, srv63109996=83, srv1113172680=5, srv1096640647=4, srv1647133133=34, srv1933466233=53, srv1594578614=31, srv485456238=71, srv1066933731=3, srv63735580=86, srv1120850972=7, srv1737214513=40, srv73946310=90, srv2116194978=62, srv1859425244=47, srv612949667=82, srv1355811395=21, srv1781412932=41, srv1808050684=44, srv94254210=98, srv2058563439=58, srv1630387092=33, srv1191248077=12, srv559868756=77, srv1159616674=9, srv1339880015=19, srv1148435655=8, srv478158527=70, srv1463760563=26, srv1887621165=51, srv634834518=84, srv1678600450=36, srv337478680=67, srv797476008=93, srv168179345=37, srv1792104435=43, srv554170984=76, srv575984120=79, srv651289261=88, srv1904065509=52, srv48990728=72, srv790864539=92, srv2066111369=59, srv497238446=74, srv38135197=68, srv645588265=87, srv2029315690=56, srv1217294750=14, srv1810451007=45, srv692236010=89, srv2022677446=55, srv496085330=73, srv937223888=97, srv576744135=81, srv2091509778=61, srv104424510=1, srv1430192927=25, srv922652738=96, srv1879764726=48, srv1484537588=27, srv1501772242=29, srv570989013=78, srv1166593989=10, srv1189954606=11, srv1310622926=16, srv170514437=39, srv1833703773=46, srv265376845=64, srv1415646824=23, srv273872085=65, srv576488542=80, srv2085400472=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:42,965 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:42,966 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:42,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:42,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:42,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:42,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-28 08:55:42,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-28 08:55:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-28 08:55:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-28 08:55:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-28 08:55:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-28 08:55:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-28 08:55:42,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-28 08:55:42,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-28 08:55:42,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-28 08:55:42,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-28 08:55:42,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-28 08:55:42,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-28 08:55:42,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-28 08:55:42,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-28 08:55:42,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-28 08:55:42,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-28 08:55:42,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-28 08:55:42,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-28 08:55:42,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-28 08:55:42,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-28 08:55:42,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-28 08:55:42,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-28 08:55:42,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-28 08:55:42,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-28 08:55:42,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-28 08:55:42,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-28 08:55:42,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-28 08:55:42,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-28 08:55:42,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-28 08:55:42,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-28 08:55:42,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-28 08:55:42,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-28 08:55:42,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-28 08:55:42,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-28 08:55:42,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-28 08:55:42,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-28 08:55:42,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-28 08:55:42,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-28 08:55:42,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-28 08:55:42,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-28 08:55:42,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-28 08:55:42,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-28 08:55:42,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-28 08:55:42,979 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-28 08:55:42,980 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-28 08:55:42,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-28 08:55:42,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-28 08:55:42,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-28 08:55:42,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-28 08:55:42,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-28 08:55:42,981 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-28 08:55:42,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-28 08:55:42,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-28 08:55:42,982 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-28 08:55:43,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:43,307 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-28 08:55:43,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-28 08:55:43,695 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 667 ms to try 5000 different iterations.  Found a solution that moves 4701 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.1813088707818509. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040830937804320594, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05248910955401404, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5223333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040830937804320594, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:43,719 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1422898035:82 , srv576488542:82 , srv63512975:82 , srv103028325:83 , srv1339880015:83 , srv1413854968:83 , srv1904065509:83 , srv2022677446:83 , srv2048469696:83 , srv104755312:84 , srv1066933731:84 , srv134036069:84 , srv1630387092:84 , srv1647133133:84 , srv168179345:84 , srv1880949237:84 , srv1887621165:84 , srv1933466233:84 , srv485456238:84 , srv570989013:84 , srv575984120:84 , srv63735580:84 , srv645588265:84 , srv73946310:84 , srv86089871:84 , srv104424510:85 , srv1096640647:85 , srv1120850972:85 , srv1189954606:85 , srv1355811395:85 , srv1463760563:85 , srv1491216967:85 , srv1668828601:85 , srv1694813821:85 , srv1790482521:85 , srv2058563439:85 , srv2116194978:85 , srv265376845:85 , srv392414637:85 , srv612949667:85 , srv1159616674:86 , srv1166593989:86 , srv1303029047:86 , srv1313144862:86 , srv1501772242:86 , srv170514437:86 , srv1808050684:86 , srv1833703773:86 , srv1885241548:86 , srv2085400472:86 , srv227698348:86 , srv337478680:86 , srv478158527:86 , srv48990728:86 , srv496085330:86 , srv497238446:86 , srv554170984:86 , srv559868756:86 , srv82750588:86 , srv986977854:86 , srv1415646824:87 , srv1484537588:87 , srv1525570067:87 , srv1737214513:87 , srv1859425244:87 , srv2029315690:87 , srv2066111369:87 , srv550648444:87 , srv740066283:87 , srv797476008:87 , srv94254210:87 , srv1148435655:88 , srv1216661652:88 , srv1430192927:88 , srv1781412932:88 , srv1792104435:88 , srv1950731376:88 , srv576744135:88 , srv692236010:88 , srv1115857553:89 , srv1191248077:89 , srv1217294750:89 , srv1310622926:89 , srv1594578614:89 , srv2091509778:89 , srv38135197:89 , srv63109996:89 , srv651289261:89 , srv273872085:90 , srv334862856:90 , srv790864539:90 , srv937223888:90 , srv1113172680:91 , srv1312809948:91 , srv1879764726:91 , srv1678600450:92 , srv1810451007:92 , srv922652738:92 , srv634834518:93 , srv1599355320:453 }
2023-04-28 08:55:43,728 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:43,730 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv82750588=94, srv2048469696=57, srv986977854=99, srv1525570067=30, srv227698348=63, srv1303029047=15, srv1115857553=6, srv392414637=69, srv1885241548=50, srv1790482521=42, srv1950731376=54, srv550648444=75, srv1312809948=17, srv103028325=0, srv1668828601=35, srv134036069=20, srv1694813821=38, srv1599355320=32, srv1880949237=49, srv1313144862=18, srv334862856=66, srv104755312=2, srv740066283=91, srv86089871=95, srv1216661652=13, srv1491216967=28, srv1413854968=22, srv63512975=85, srv1422898035=24, srv63109996=83, srv1113172680=5, srv1096640647=4, srv1647133133=34, srv1933466233=53, srv1594578614=31, srv485456238=71, srv1066933731=3, srv63735580=86, srv1120850972=7, srv1737214513=40, srv73946310=90, srv2116194978=62, srv1859425244=47, srv612949667=82, srv1355811395=21, srv1781412932=41, srv1808050684=44, srv94254210=98, srv2058563439=58, srv1630387092=33, srv1191248077=12, srv559868756=77, srv1159616674=9, srv1339880015=19, srv1148435655=8, srv478158527=70, srv1463760563=26, srv1887621165=51, srv634834518=84, srv1678600450=36, srv337478680=67, srv797476008=93, srv168179345=37, srv1792104435=43, srv554170984=76, srv575984120=79, srv651289261=88, srv1904065509=52, srv48990728=72, srv790864539=92, srv2066111369=59, srv497238446=74, srv38135197=68, srv645588265=87, srv2029315690=56, srv1217294750=14, srv1810451007=45, srv692236010=89, srv2022677446=55, srv496085330=73, srv937223888=97, srv576744135=81, srv2091509778=61, srv104424510=1, srv1430192927=25, srv922652738=96, srv1879764726=48, srv1484537588=27, srv1501772242=29, srv570989013=78, srv1166593989=10, srv1189954606=11, srv1310622926=16, srv170514437=39, srv1833703773=46, srv265376845=64, srv1415646824=23, srv273872085=65, srv576488542=80, srv2085400472=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:43,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:43,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:43,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:43,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:43,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:43,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:43,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:43,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:43,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:43,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:43,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:43,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:43,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:43,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:43,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:43,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:43,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:43,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:43,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:43,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:43,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:43,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:43,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:43,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:43,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:43,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:43,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:43,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:43,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:43,773 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-28 08:55:43,774 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-28 08:55:43,775 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-28 08:55:43,776 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-28 08:55:43,777 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-28 08:55:43,778 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-28 08:55:43,779 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-28 08:55:43,780 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-28 08:55:43,781 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-28 08:55:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-28 08:55:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-28 08:55:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-28 08:55:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-28 08:55:43,782 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-28 08:55:43,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040830937804320594, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05248910955401404, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040830937804320594, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:43,949 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-28 08:55:43,949 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.18127595367629948, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.040830937804320594, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05248910955401404, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.040830937804320594, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17675, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24125, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-28 08:55:44,353 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 548 ms to try 5000 different iterations.  Found a solution that moves 2636 regions; Going from a computed imbalance of 0.18127595367629948 to a new imbalance of 0.016515235149653373. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022067926742154423); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029545756710878234, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29288888888888887, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022067926742154423); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04725, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:44,372 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1879764726:79 , srv922652738:79 , srv1312809948:80 , srv1113172680:81 , srv63109996:81 , srv634834518:81 , srv1216661652:82 , srv1594578614:82 , srv1792104435:82 , srv1810451007:82 , srv273872085:82 , srv790864539:82 , srv1115857553:83 , srv1148435655:83 , srv1310622926:83 , srv1430192927:83 , srv1678600450:83 , srv1859425244:83 , srv1885241548:83 , srv334862856:83 , srv651289261:83 , srv937223888:83 , srv1781412932:84 , srv2091509778:84 , srv576744135:84 , srv1120850972:85 , srv1191248077:85 , srv1217294750:85 , srv1463760563:85 , srv1491216967:85 , srv1630387092:85 , srv1833703773:85 , srv1950731376:85 , srv2058563439:85 , srv2085400472:85 , srv337478680:85 , srv38135197:85 , srv478158527:85 , srv497238446:85 , srv550648444:85 , srv554170984:85 , srv559868756:85 , srv63735580:85 , srv797476008:85 , srv94254210:85 , srv986977854:85 , srv104755312:86 , srv1159616674:86 , srv1166593989:86 , srv1355811395:86 , srv1484537588:86 , srv1501772242:86 , srv1694813821:86 , srv1737214513:86 , srv1790482521:86 , srv1880949237:86 , srv2066111369:86 , srv227698348:86 , srv570989013:86 , srv645588265:86 , srv740066283:86 , srv104424510:87 , srv1066933731:87 , srv1096640647:87 , srv1189954606:87 , srv1313144862:87 , srv1525570067:87 , srv1668828601:87 , srv1887621165:87 , srv1933466233:87 , srv2029315690:87 , srv48990728:87 , srv496085330:87 , srv612949667:87 , srv692236010:87 , srv73946310:87 , srv82750588:87 , srv86089871:87 , srv103028325:88 , srv134036069:88 , srv170514437:88 , srv1808050684:88 , srv265376845:88 , srv392414637:88 , srv575984120:88 , srv576488542:88 , srv1303029047:89 , srv1415646824:89 , srv1422898035:89 , srv168179345:89 , srv1904065509:89 , srv63512975:89 , srv1413854968:90 , srv1647133133:90 , srv2116194978:90 , srv485456238:90 , srv2048469696:91 , srv1339880015:92 , srv2022677446:93 , srv1599355320:517 }
2023-04-28 08:55:44,378 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:44,380 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv82750588=94, srv2048469696=57, srv986977854=99, srv1525570067=30, srv227698348=63, srv1303029047=15, srv1115857553=6, srv392414637=69, srv1885241548=50, srv1790482521=42, srv1950731376=54, srv550648444=75, srv1312809948=17, srv103028325=0, srv1668828601=35, srv134036069=20, srv1694813821=38, srv1599355320=32, srv1880949237=49, srv1313144862=18, srv334862856=66, srv104755312=2, srv740066283=91, srv86089871=95, srv1216661652=13, srv1491216967=28, srv1413854968=22, srv63512975=85, srv1422898035=24, srv63109996=83, srv1113172680=5, srv1096640647=4, srv1647133133=34, srv1933466233=53, srv1594578614=31, srv485456238=71, srv1066933731=3, srv63735580=86, srv1120850972=7, srv1737214513=40, srv73946310=90, srv2116194978=62, srv1859425244=47, srv612949667=82, srv1355811395=21, srv1781412932=41, srv1808050684=44, srv94254210=98, srv2058563439=58, srv1630387092=33, srv1191248077=12, srv559868756=77, srv1159616674=9, srv1339880015=19, srv1148435655=8, srv478158527=70, srv1463760563=26, srv1887621165=51, srv634834518=84, srv1678600450=36, srv337478680=67, srv797476008=93, srv168179345=37, srv1792104435=43, srv554170984=76, srv575984120=79, srv651289261=88, srv1904065509=52, srv48990728=72, srv790864539=92, srv2066111369=59, srv497238446=74, srv38135197=68, srv645588265=87, srv2029315690=56, srv1217294750=14, srv1810451007=45, srv692236010=89, srv2022677446=55, srv496085330=73, srv937223888=97, srv576744135=81, srv2091509778=61, srv104424510=1, srv1430192927=25, srv922652738=96, srv1879764726=48, srv1484537588=27, srv1501772242=29, srv570989013=78, srv1166593989=10, srv1189954606=11, srv1310622926=16, srv170514437=39, srv1833703773=46, srv265376845=64, srv1415646824=23, srv273872085=65, srv576488542=80, srv2085400472=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:44,409 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:44,410 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:44,411 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:44,412 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-28 08:55:44,413 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-28 08:55:44,414 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-28 08:55:44,415 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-28 08:55:44,416 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-28 08:55:44,417 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-28 08:55:44,418 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-28 08:55:44,419 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-28 08:55:44,420 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-28 08:55:44,571 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022067926742154423); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029545756710878234, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022067926742154423); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04725, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:44,572 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-28 08:55:44,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016496777483149754, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.022067926742154423); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.029545756710878234, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.022067926742154423); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013333333333333334); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04725, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-28 08:55:44,975 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 529 ms to try 5000 different iterations.  Found a solution that moves 1013 regions; Going from a computed imbalance of 0.016496777483149754 to a new imbalance of 7.711123300976299E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006538231594101115); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006067336330376245); MoveCostFunction : (multiplier=7.0, imbalance=0.11255555555555556, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006538231594101115); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:44,978 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1415646824:81 , srv1678600450:82 , srv2022677446:82 , srv2116194978:82 , srv1191248077:83 , srv1217294750:83 , srv1303029047:83 , srv1339880015:83 , srv1525570067:83 , srv1647133133:83 , srv1808050684:83 , srv2048469696:83 , srv485456238:83 , srv692236010:83 , srv1313144862:84 , srv170514437:84 , srv1737214513:84 , srv1810451007:84 , srv1950731376:84 , srv2029315690:84 , srv265376845:84 , srv334862856:84 , srv38135197:84 , srv48990728:84 , srv496085330:84 , srv634834518:84 , srv82750588:84 , srv937223888:84 , srv1096640647:85 , srv1113172680:85 , srv1115857553:85 , srv1159616674:85 , srv1310622926:85 , srv134036069:85 , srv1413854968:85 , srv1430192927:85 , srv1501772242:85 , srv1668828601:85 , srv168179345:85 , srv1904065509:85 , srv2066111369:85 , srv2091509778:85 , srv273872085:85 , srv392414637:85 , srv575984120:85 , srv612949667:85 , srv740066283:85 , srv790864539:85 , srv103028325:86 , srv104424510:86 , srv1148435655:86 , srv1166593989:86 , srv1189954606:86 , srv1312809948:86 , srv1355811395:86 , srv1422898035:86 , srv1484537588:86 , srv1594578614:86 , srv1694813821:86 , srv1781412932:86 , srv1790482521:86 , srv1887621165:86 , srv227698348:86 , srv337478680:86 , srv497238446:86 , srv550648444:86 , srv554170984:86 , srv559868756:86 , srv576744135:86 , srv651289261:86 , srv797476008:86 , srv922652738:86 , srv94254210:86 , srv104755312:87 , srv1066933731:87 , srv1120850972:87 , srv1463760563:87 , srv1792104435:87 , srv1833703773:87 , srv1879764726:87 , srv1880949237:87 , srv1933466233:87 , srv2085400472:87 , srv478158527:87 , srv570989013:87 , srv576488542:87 , srv63109996:87 , srv63512975:87 , srv645588265:87 , srv73946310:87 , srv986977854:87 , srv1216661652:88 , srv1491216967:88 , srv1859425244:88 , srv1885241548:88 , srv2058563439:88 , srv63735580:88 , srv86089871:88 , srv1630387092:89 , srv1599355320:546 }
2023-04-28 08:55:44,986 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:44,987 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv82750588=94, srv2048469696=57, srv986977854=99, srv1525570067=30, srv227698348=63, srv1303029047=15, srv1115857553=6, srv392414637=69, srv1885241548=50, srv1790482521=42, srv1950731376=54, srv550648444=75, srv1312809948=17, srv103028325=0, srv1668828601=35, srv134036069=20, srv1694813821=38, srv1599355320=32, srv1880949237=49, srv1313144862=18, srv334862856=66, srv104755312=2, srv740066283=91, srv86089871=95, srv1216661652=13, srv1491216967=28, srv1413854968=22, srv63512975=85, srv1422898035=24, srv63109996=83, srv1113172680=5, srv1096640647=4, srv1647133133=34, srv1933466233=53, srv1594578614=31, srv485456238=71, srv1066933731=3, srv63735580=86, srv1120850972=7, srv1737214513=40, srv73946310=90, srv2116194978=62, srv1859425244=47, srv612949667=82, srv1355811395=21, srv1781412932=41, srv1808050684=44, srv94254210=98, srv2058563439=58, srv1630387092=33, srv1191248077=12, srv559868756=77, srv1159616674=9, srv1339880015=19, srv1148435655=8, srv478158527=70, srv1463760563=26, srv1887621165=51, srv634834518=84, srv1678600450=36, srv337478680=67, srv797476008=93, srv168179345=37, srv1792104435=43, srv554170984=76, srv575984120=79, srv651289261=88, srv1904065509=52, srv48990728=72, srv790864539=92, srv2066111369=59, srv497238446=74, srv38135197=68, srv645588265=87, srv2029315690=56, srv1217294750=14, srv1810451007=45, srv692236010=89, srv2022677446=55, srv496085330=73, srv937223888=97, srv576744135=81, srv2091509778=61, srv104424510=1, srv1430192927=25, srv922652738=96, srv1879764726=48, srv1484537588=27, srv1501772242=29, srv570989013=78, srv1166593989=10, srv1189954606=11, srv1310622926=16, srv170514437=39, srv1833703773=46, srv265376845=64, srv1415646824=23, srv273872085=65, srv576488542=80, srv2085400472=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:45,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:45,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:45,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-28 08:55:45,009 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-28 08:55:45,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-28 08:55:45,011 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-28 08:55:45,012 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-28 08:55:45,013 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-28 08:55:45,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-28 08:55:45,015 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-28 08:55:45,016 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-28 08:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-28 08:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-28 08:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-28 08:55:45,017 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-28 08:55:45,162 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006538231594101115); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006067336330376245); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006538231594101115); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:45,162 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-28 08:55:45,162 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=7.64019152492105E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006538231594101115); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006067336330376245); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006538231594101115); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.005333333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-28 08:55:45,695 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 658 ms to try 5000 different iterations.  Found a solution that moves 285 regions; Going from a computed imbalance of 7.64019152492105E-4 to a new imbalance of 2.937954760868148E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.03166666666666667, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:45,698 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1413854968:83 , srv1484537588:83 , srv1491216967:83 , srv2029315690:83 , srv86089871:83 , srv104424510:84 , srv1066933731:84 , srv1120850972:84 , srv1166593989:84 , srv1189954606:84 , srv1303029047:84 , srv1339880015:84 , srv134036069:84 , srv1630387092:84 , srv168179345:84 , srv170514437:84 , srv1737214513:84 , srv1781412932:84 , srv1810451007:84 , srv1833703773:84 , srv1859425244:84 , srv1933466233:84 , srv2058563439:84 , srv2066111369:84 , srv2085400472:84 , srv2091509778:84 , srv227698348:84 , srv38135197:84 , srv392414637:84 , srv478158527:84 , srv550648444:84 , srv576488542:84 , srv576744135:84 , srv63512975:84 , srv651289261:84 , srv73946310:84 , srv740066283:84 , srv797476008:84 , srv922652738:84 , srv94254210:84 , srv103028325:85 , srv104755312:85 , srv1096640647:85 , srv1113172680:85 , srv1115857553:85 , srv1148435655:85 , srv1159616674:85 , srv1191248077:85 , srv1216661652:85 , srv1217294750:85 , srv1310622926:85 , srv1312809948:85 , srv1313144862:85 , srv1355811395:85 , srv1415646824:85 , srv1422898035:85 , srv1463760563:85 , srv1501772242:85 , srv1525570067:85 , srv1594578614:85 , srv1647133133:85 , srv1668828601:85 , srv1678600450:85 , srv1694813821:85 , srv1790482521:85 , srv1808050684:85 , srv1879764726:85 , srv1880949237:85 , srv1885241548:85 , srv1887621165:85 , srv1904065509:85 , srv1950731376:85 , srv2022677446:85 , srv2048469696:85 , srv2116194978:85 , srv265376845:85 , srv273872085:85 , srv334862856:85 , srv337478680:85 , srv485456238:85 , srv48990728:85 , srv496085330:85 , srv497238446:85 , srv554170984:85 , srv559868756:85 , srv570989013:85 , srv575984120:85 , srv634834518:85 , srv63735580:85 , srv645588265:85 , srv692236010:85 , srv790864539:85 , srv82750588:85 , srv937223888:85 , srv986977854:85 , srv1430192927:86 , srv1792104435:86 , srv612949667:86 , srv63109996:86 , srv1599355320:626 }
2023-04-28 08:55:45,706 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:45,719 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv82750588=94, srv2048469696=57, srv986977854=99, srv1525570067=30, srv227698348=63, srv1303029047=15, srv1115857553=6, srv392414637=69, srv1885241548=50, srv1790482521=42, srv1950731376=54, srv550648444=75, srv1312809948=17, srv103028325=0, srv1668828601=35, srv134036069=20, srv1694813821=38, srv1599355320=32, srv1880949237=49, srv1313144862=18, srv334862856=66, srv104755312=2, srv740066283=91, srv86089871=95, srv1216661652=13, srv1491216967=28, srv1413854968=22, srv63512975=85, srv1422898035=24, srv63109996=83, srv1113172680=5, srv1096640647=4, srv1647133133=34, srv1933466233=53, srv1594578614=31, srv485456238=71, srv1066933731=3, srv63735580=86, srv1120850972=7, srv1737214513=40, srv73946310=90, srv2116194978=62, srv1859425244=47, srv612949667=82, srv1355811395=21, srv1781412932=41, srv1808050684=44, srv94254210=98, srv2058563439=58, srv1630387092=33, srv1191248077=12, srv559868756=77, srv1159616674=9, srv1339880015=19, srv1148435655=8, srv478158527=70, srv1463760563=26, srv1887621165=51, srv634834518=84, srv1678600450=36, srv337478680=67, srv797476008=93, srv168179345=37, srv1792104435=43, srv554170984=76, srv575984120=79, srv651289261=88, srv1904065509=52, srv48990728=72, srv790864539=92, srv2066111369=59, srv497238446=74, srv38135197=68, srv645588265=87, srv2029315690=56, srv1217294750=14, srv1810451007=45, srv692236010=89, srv2022677446=55, srv496085330=73, srv937223888=97, srv576744135=81, srv2091509778=61, srv104424510=1, srv1430192927=25, srv922652738=96, srv1879764726=48, srv1484537588=27, srv1501772242=29, srv570989013=78, srv1166593989=10, srv1189954606=11, srv1310622926=16, srv170514437=39, srv1833703773=46, srv265376845=64, srv1415646824=23, srv273872085=65, srv576488542=80, srv2085400472=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:45,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:45,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:45,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:45,737 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:45,738 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:45,739 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-28 08:55:45,740 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-28 08:55:45,741 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-28 08:55:45,742 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-28 08:55:45,743 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-28 08:55:45,744 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-28 08:55:45,745 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-28 08:55:45,746 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-28 08:55:45,747 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-28 08:55:45,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-28 08:55:45,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-28 08:55:45,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-28 08:55:45,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-28 08:55:45,748 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-28 08:55:45,872 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:45,872 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-04-28 08:55:45,873 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.7383934955596984E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0022721660362024225); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-04-28 08:55:46,116 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 353 ms to try 5000 different iterations.  Found a solution that moves 42 regions; Going from a computed imbalance of 2.7383934955596984E-5 to a new imbalance of 7.038159339549663E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.004666666666666667); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:46,117 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1216661652:83 , srv1792104435:83 , srv2022677446:83 , srv612949667:83 , srv63109996:83 , srv634834518:83 , srv692236010:83 , srv986977854:83 , srv103028325:84 , srv104424510:84 , srv104755312:84 , srv1066933731:84 , srv1096640647:84 , srv1113172680:84 , srv1115857553:84 , srv1148435655:84 , srv1159616674:84 , srv1166593989:84 , srv1189954606:84 , srv1191248077:84 , srv1217294750:84 , srv1303029047:84 , srv1310622926:84 , srv1312809948:84 , srv1313144862:84 , srv1339880015:84 , srv1355811395:84 , srv1415646824:84 , srv1422898035:84 , srv1430192927:84 , srv1463760563:84 , srv1484537588:84 , srv1501772242:84 , srv1525570067:84 , srv1594578614:84 , srv1630387092:84 , srv1647133133:84 , srv1668828601:84 , srv1678600450:84 , srv168179345:84 , srv1694813821:84 , srv170514437:84 , srv1781412932:84 , srv1790482521:84 , srv1808050684:84 , srv1810451007:84 , srv1833703773:84 , srv1859425244:84 , srv1879764726:84 , srv1880949237:84 , srv1885241548:84 , srv1887621165:84 , srv1904065509:84 , srv1933466233:84 , srv1950731376:84 , srv2048469696:84 , srv2058563439:84 , srv2066111369:84 , srv2085400472:84 , srv2091509778:84 , srv2116194978:84 , srv227698348:84 , srv265376845:84 , srv273872085:84 , srv334862856:84 , srv337478680:84 , srv38135197:84 , srv392414637:84 , srv478158527:84 , srv485456238:84 , srv48990728:84 , srv496085330:84 , srv497238446:84 , srv550648444:84 , srv554170984:84 , srv559868756:84 , srv570989013:84 , srv575984120:84 , srv576744135:84 , srv63512975:84 , srv63735580:84 , srv645588265:84 , srv651289261:84 , srv73946310:84 , srv740066283:84 , srv790864539:84 , srv797476008:84 , srv82750588:84 , srv86089871:84 , srv937223888:84 , srv94254210:84 , srv1120850972:85 , srv134036069:85 , srv1413854968:85 , srv1491216967:85 , srv1737214513:85 , srv2029315690:85 , srv576488542:85 , srv922652738:85 , srv1599355320:684 }
2023-04-28 08:55:46,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:46,125 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv82750588=94, srv2048469696=57, srv986977854=99, srv1525570067=30, srv227698348=63, srv1303029047=15, srv1115857553=6, srv392414637=69, srv1885241548=50, srv1790482521=42, srv1950731376=54, srv550648444=75, srv1312809948=17, srv103028325=0, srv1668828601=35, srv134036069=20, srv1694813821=38, srv1599355320=32, srv1880949237=49, srv1313144862=18, srv334862856=66, srv104755312=2, srv740066283=91, srv86089871=95, srv1216661652=13, srv1491216967=28, srv1413854968=22, srv63512975=85, srv1422898035=24, srv63109996=83, srv1113172680=5, srv1096640647=4, srv1647133133=34, srv1933466233=53, srv1594578614=31, srv485456238=71, srv1066933731=3, srv63735580=86, srv1120850972=7, srv1737214513=40, srv73946310=90, srv2116194978=62, srv1859425244=47, srv612949667=82, srv1355811395=21, srv1781412932=41, srv1808050684=44, srv94254210=98, srv2058563439=58, srv1630387092=33, srv1191248077=12, srv559868756=77, srv1159616674=9, srv1339880015=19, srv1148435655=8, srv478158527=70, srv1463760563=26, srv1887621165=51, srv634834518=84, srv1678600450=36, srv337478680=67, srv797476008=93, srv168179345=37, srv1792104435=43, srv554170984=76, srv575984120=79, srv651289261=88, srv1904065509=52, srv48990728=72, srv790864539=92, srv2066111369=59, srv497238446=74, srv38135197=68, srv645588265=87, srv2029315690=56, srv1217294750=14, srv1810451007=45, srv692236010=89, srv2022677446=55, srv496085330=73, srv937223888=97, srv576744135=81, srv2091509778=61, srv104424510=1, srv1430192927=25, srv922652738=96, srv1879764726=48, srv1484537588=27, srv1501772242=29, srv570989013=78, srv1166593989=10, srv1189954606=11, srv1310622926=16, srv170514437=39, srv1833703773=46, srv265376845=64, srv1415646824=23, srv273872085=65, srv576488542=80, srv2085400472=60} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:46,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:46,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:46,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:46,143 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:46,144 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:46,145 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-04-28 08:55:46,146 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-04-28 08:55:46,147 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-04-28 08:55:46,148 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-04-28 08:55:46,149 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-04-28 08:55:46,150 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-04-28 08:55:46,151 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-04-28 08:55:46,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-04-28 08:55:46,152 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-04-28 08:55:46,296 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=6.744069053831948E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0014982219165849825); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:46,297 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1216661652:83 , srv1792104435:83 , srv2022677446:83 , srv612949667:83 , srv63109996:83 , srv634834518:83 , srv692236010:83 , srv986977854:83 , srv103028325:84 , srv104424510:84 , srv104755312:84 , srv1066933731:84 , srv1096640647:84 , srv1113172680:84 , srv1115857553:84 , srv1148435655:84 , srv1159616674:84 , srv1166593989:84 , srv1189954606:84 , srv1191248077:84 , srv1217294750:84 , srv1303029047:84 , srv1310622926:84 , srv1312809948:84 , srv1313144862:84 , srv1339880015:84 , srv1355811395:84 , srv1415646824:84 , srv1422898035:84 , srv1430192927:84 , srv1463760563:84 , srv1484537588:84 , srv1501772242:84 , srv1525570067:84 , srv1594578614:84 , srv1630387092:84 , srv1647133133:84 , srv1668828601:84 , srv1678600450:84 , srv168179345:84 , srv1694813821:84 , srv170514437:84 , srv1781412932:84 , srv1790482521:84 , srv1808050684:84 , srv1810451007:84 , srv1833703773:84 , srv1859425244:84 , srv1879764726:84 , srv1880949237:84 , srv1885241548:84 , srv1887621165:84 , srv1904065509:84 , srv1933466233:84 , srv1950731376:84 , srv2048469696:84 , srv2058563439:84 , srv2066111369:84 , srv2085400472:84 , srv2091509778:84 , srv2116194978:84 , srv227698348:84 , srv265376845:84 , srv273872085:84 , srv334862856:84 , srv337478680:84 , srv38135197:84 , srv392414637:84 , srv478158527:84 , srv485456238:84 , srv48990728:84 , srv496085330:84 , srv497238446:84 , srv550648444:84 , srv554170984:84 , srv559868756:84 , srv570989013:84 , srv575984120:84 , srv576744135:84 , srv63512975:84 , srv63735580:84 , srv645588265:84 , srv651289261:84 , srv73946310:84 , srv740066283:84 , srv790864539:84 , srv797476008:84 , srv82750588:84 , srv86089871:84 , srv937223888:84 , srv94254210:84 , srv1120850972:85 , srv134036069:85 , srv1413854968:85 , srv1491216967:85 , srv1737214513:85 , srv2029315690:85 , srv576488542:85 , srv922652738:85 , srv1599355320:684 }
2023-04-28 08:55:46,327 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=396 (was 396), ProcessCount=170 (was 170), AvailableMemoryMB=6459 (was 6937)
2023-04-28 08:55:46,337 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=396, ProcessCount=170, AvailableMemoryMB=6459
2023-04-28 08:55:46,338 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-04-28 08:55:46,338 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-04-28 08:55:46,353 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1151932134:84 , srv1424437900:84 , srv142717667:84 , srv1490054577:84 , srv1492578343:84 , srv1530480172:84 , srv1547737811:84 , srv1565923189:84 , srv1583100573:84 , srv1659446757:84 , srv1696870460:84 , srv1873056921:84 , srv1969294977:84 , srv1989568223:84 , srv2060666140:84 , srv2283356:84 , srv318596652:84 , srv331763094:84 , srv332094417:84 , srv360113578:84 , srv496178573:84 , srv572480052:84 , srv587411706:84 , srv625535630:84 , srv791607695:84 , srv801990913:84 , srv865321422:84 , srv956079394:84 , srv976512378:84 , srv1383931753:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-04-28 08:55:46,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:55:46,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv318596652=17, srv1583100573=9, srv1490054577=4, srv1424437900=2, srv1989568223=14, srv360113578=20, srv956079394=28, srv1492578343=5, srv1151932134=0, srv1659446757=10, srv1530480172=6, srv801990913=26, srv1873056921=12, srv625535630=24, srv2060666140=15, srv976512378=29, srv2283356=16, srv865321422=27, srv142717667=3, srv791607695=25, srv331763094=18, srv1547737811=7, srv1383931753=1, srv496178573=21, srv587411706=23, srv1565923189=8, srv1969294977=13, srv332094417=19, srv572480052=22, srv1696870460=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:55:46,371 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:55:46,372 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:55:46,373 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-28 08:55:46,401 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1257314744556643, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:55:46,402 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912589219034779, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1257314744556643, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-04-28 08:57:46,378 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5812131 different iterations.  Found a solution that moves 1845 regions; Going from a computed imbalance of 0.9912589219034779 to a new imbalance of 4.306322040866546E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6833333333333333, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:57:46,384 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1151932134:90 , srv1383931753:90 , srv1424437900:90 , srv142717667:90 , srv1490054577:90 , srv1492578343:90 , srv1530480172:90 , srv1547737811:90 , srv1565923189:90 , srv1583100573:90 , srv1659446757:90 , srv1696870460:90 , srv1873056921:90 , srv1969294977:90 , srv1989568223:90 , srv2060666140:90 , srv2283356:90 , srv318596652:90 , srv331763094:90 , srv332094417:90 , srv360113578:90 , srv496178573:90 , srv572480052:90 , srv587411706:90 , srv625535630:90 , srv791607695:90 , srv801990913:90 , srv865321422:90 , srv956079394:90 , srv976512378:90 }
2023-04-28 08:57:46,385 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-04-28 08:57:46,386 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv318596652=17, srv1583100573=9, srv1490054577=4, srv1424437900=2, srv1989568223=14, srv360113578=20, srv956079394=28, srv1492578343=5, srv1151932134=0, srv1659446757=10, srv1530480172=6, srv801990913=26, srv1873056921=12, srv625535630=24, srv2060666140=15, srv976512378=29, srv2283356=16, srv865321422=27, srv142717667=3, srv791607695=25, srv331763094=18, srv1547737811=7, srv1383931753=1, srv496178573=21, srv587411706=23, srv1565923189=8, srv1969294977=13, srv332094417=19, srv572480052=22, srv1696870460=11} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-04-28 08:57:46,389 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-04-28 08:57:46,390 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-04-28 08:57:46,402 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-04-28 08:57:46,402 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1151932134:90 , srv1383931753:90 , srv1424437900:90 , srv142717667:90 , srv1490054577:90 , srv1492578343:90 , srv1530480172:90 , srv1547737811:90 , srv1565923189:90 , srv1583100573:90 , srv1659446757:90 , srv1696870460:90 , srv1873056921:90 , srv1969294977:90 , srv1989568223:90 , srv2060666140:90 , srv2283356:90 , srv318596652:90 , srv331763094:90 , srv332094417:90 , srv360113578:90 , srv496178573:90 , srv572480052:90 , srv587411706:90 , srv625535630:90 , srv791607695:90 , srv801990913:90 , srv865321422:90 , srv956079394:90 , srv976512378:90 }
2023-04-28 08:57:46,414 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=395 (was 396), ProcessCount=169 (was 170), AvailableMemoryMB=6374 (was 6459)