2023-03-10 12:54:15,347 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-03-10 12:54:15,671 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-03-10 12:54:15,786 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-03-10 12:54:15,846 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-10 12:54:15,854 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-10 12:54:15,866 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=170, AvailableMemoryMB=6032
2023-03-10 12:54:15,869 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-10 12:54:15,869 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-03-10 12:54:16,001 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv100551892:84 , srv1025425038:84 , srv1095858553:84 , srv1097348085:84 , srv1099353935:84 , srv1131609332:84 , srv1141611690:84 , srv1152097368:84 , srv115580900:84 , srv1164155242:84 , srv1167220210:84 , srv1168993528:84 , srv116979828:84 , srv117002844:84 , srv1180992657:84 , srv1181911679:84 , srv1289358191:84 , srv1293153957:84 , srv131508293:84 , srv1326106814:84 , srv1431219816:84 , srv1444687027:84 , srv1459065897:84 , srv1460782924:84 , srv1504807184:84 , srv1510412012:84 , srv1513571950:84 , srv1520663653:84 , srv1528891442:84 , srv1572573157:84 , srv1602917888:84 , srv1608387864:84 , srv1624426746:84 , srv1646790675:84 , srv165397939:84 , srv1670345235:84 , srv1694181643:84 , srv171445303:84 , srv173006868:84 , srv1755666932:84 , srv175796375:84 , srv1769351916:84 , srv1770751599:84 , srv1800694279:84 , srv1811372937:84 , srv1817146903:84 , srv1839642147:84 , srv18547335:84 , srv1894224832:84 , srv1913073400:84 , srv1919822980:84 , srv1927808531:84 , srv1996530885:84 , srv2010539834:84 , srv2010968701:84 , srv2048184643:84 , srv2049376735:84 , srv2085004527:84 , srv2087352896:84 , srv2139978042:84 , srv2146461733:84 , srv227592999:84 , srv260742302:84 , srv274214721:84 , srv288449217:84 , srv292678086:84 , srv318137291:84 , srv320743146:84 , srv335428563:84 , srv406165961:84 , srv408547582:84 , srv432571802:84 , srv437217299:84 , srv469184640:84 , srv491866102:84 , srv492552268:84 , srv494849059:84 , srv513352721:84 , srv514136320:84 , srv560480358:84 , srv568887862:84 , srv581892405:84 , srv60199277:84 , srv608392041:84 , srv615977597:84 , srv658839701:84 , srv697893439:84 , srv701080180:84 , srv705407629:84 , srv718344655:84 , srv74925929:84 , srv783264483:84 , srv789702934:84 , srv817755986:84 , srv819851503:84 , srv848334531:84 , srv87239816:84 , srv879178437:84 , srv919680812:84 , srv1372387710:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-03-10 12:54:16,008 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:16,014 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1608387864=32, srv1168993528=11, srv1913073400=50, srv491866102=75, srv87239816=97, srv1141611690=6, srv608392041=84, srv175796375=41, srv131508293=18, srv227592999=62, srv1770751599=43, srv1097348085=3, srv2085004527=58, srv2010968701=55, srv1152097368=7, srv1431219816=21, srv1180992657=14, srv1817146903=46, srv1894224832=49, srv2087352896=59, srv783264483=92, srv718344655=90, srv1459065897=23, srv469184640=74, srv2049376735=57, srv494849059=77, srv1528891442=29, srv1513571950=27, srv318137291=67, srv1504807184=25, srv100551892=0, srv2048184643=56, srv1444687027=22, srv1326106814=19, srv581892405=82, srv919680812=99, srv1025425038=1, srv1510412012=26, srv1919822980=51, srv1811372937=45, srv1372387710=20, srv406165961=70, srv1670345235=36, srv1769351916=42, srv320743146=68, srv1181911679=15, srv173006868=39, srv819851503=95, srv432571802=72, srv1167220210=10, srv1289358191=16, srv60199277=83, srv1099353935=4, srv292678086=66, srv697893439=87, srv1572573157=30, srv1927808531=52, srv2010539834=54, srv817755986=94, srv1839642147=47, srv74925929=91, srv2139978042=60, srv1624426746=33, srv1800694279=44, srv437217299=73, srv171445303=38, srv701080180=88, srv1131609332=5, srv288449217=65, srv1460782924=24, srv1602917888=31, srv1755666932=40, srv165397939=35, srv335428563=69, srv705407629=89, srv615977597=85, srv117002844=13, srv513352721=78, srv879178437=98, srv115580900=8, srv1694181643=37, srv514136320=79, srv848334531=96, srv1520663653=28, srv18547335=48, srv260742302=63, srv1646790675=34, srv274214721=64, srv1996530885=53, srv116979828=12, srv492552268=76, srv2146461733=61, srv789702934=93, srv1095858553=2, srv1293153957=17, srv568887862=81, srv1164155242=9, srv408547582=71, srv560480358=80, srv658839701=86} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:16,111 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:16,112 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:16,113 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 12:54:16,114 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 12:54:16,115 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 12:54:16,116 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 12:54:16,117 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 12:54:16,118 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 12:54:16,119 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 12:54:16,120 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 12:54:16,314 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:16,314 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 12:54:16,314 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 12:54:16,950 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 807 ms to try 5000 different iterations.  Found a solution that moves 4716 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17717064121806084. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04075726631474507, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06488702086698189, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.524, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04075726631474507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17158333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24633333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:16,973 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv2010539834:81 , srv2010968701:81 , srv1293153957:82 , srv1510412012:82 , srv1646790675:82 , srv1913073400:82 , srv492552268:82 , srv1025425038:83 , srv1755666932:83 , srv1769351916:83 , srv292678086:83 , srv335428563:83 , srv432571802:83 , srv513352721:83 , srv60199277:83 , srv658839701:83 , srv817755986:83 , srv1431219816:84 , srv1694181643:84 , srv173006868:84 , srv1817146903:84 , srv1919822980:84 , srv2049376735:84 , srv2146461733:84 , srv318137291:84 , srv408547582:84 , srv437217299:84 , srv74925929:84 , srv1164155242:85 , srv1168993528:85 , srv1444687027:85 , srv1513571950:85 , srv1520663653:85 , srv1602917888:85 , srv1670345235:85 , srv175796375:85 , srv18547335:85 , srv514136320:85 , srv568887862:85 , srv615977597:85 , srv848334531:85 , srv879178437:85 , srv1167220210:86 , srv1180992657:86 , srv1181911679:86 , srv1504807184:86 , srv1528891442:86 , srv1624426746:86 , srv1894224832:86 , srv2139978042:86 , srv260742302:86 , srv288449217:86 , srv320743146:86 , srv581892405:86 , srv697893439:86 , srv705407629:86 , srv87239816:86 , srv1141611690:87 , srv1152097368:87 , srv115580900:87 , srv117002844:87 , srv1289358191:87 , srv1996530885:87 , srv227592999:87 , srv274214721:87 , srv783264483:87 , srv789702934:87 , srv819851503:87 , srv1095858553:88 , srv1097348085:88 , srv1131609332:88 , srv1927808531:88 , srv2087352896:88 , srv469184640:88 , srv560480358:88 , srv608392041:88 , srv718344655:88 , srv1099353935:89 , srv131508293:89 , srv165397939:89 , srv171445303:89 , srv1770751599:89 , srv494849059:89 , srv919680812:89 , srv116979828:90 , srv1326106814:90 , srv1459065897:90 , srv1572573157:90 , srv1608387864:90 , srv1800694279:90 , srv2048184643:90 , srv100551892:91 , srv1460782924:91 , srv1811372937:91 , srv701080180:92 , srv2085004527:93 , srv1839642147:94 , srv406165961:94 , srv491866102:94 , srv1372387710:452 }
2023-03-10 12:54:16,977 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:16,978 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1608387864=32, srv1168993528=11, srv1913073400=50, srv491866102=75, srv87239816=97, srv1141611690=6, srv608392041=84, srv175796375=41, srv131508293=18, srv227592999=62, srv1770751599=43, srv1097348085=3, srv2085004527=58, srv2010968701=55, srv1152097368=7, srv1431219816=21, srv1180992657=14, srv1817146903=46, srv1894224832=49, srv2087352896=59, srv783264483=92, srv718344655=90, srv1459065897=23, srv469184640=74, srv2049376735=57, srv494849059=77, srv1528891442=29, srv1513571950=27, srv318137291=67, srv1504807184=25, srv100551892=0, srv2048184643=56, srv1444687027=22, srv1326106814=19, srv581892405=82, srv919680812=99, srv1025425038=1, srv1510412012=26, srv1919822980=51, srv1811372937=45, srv1372387710=20, srv406165961=70, srv1670345235=36, srv1769351916=42, srv320743146=68, srv1181911679=15, srv173006868=39, srv819851503=95, srv432571802=72, srv1167220210=10, srv1289358191=16, srv60199277=83, srv1099353935=4, srv292678086=66, srv697893439=87, srv1572573157=30, srv1927808531=52, srv2010539834=54, srv817755986=94, srv1839642147=47, srv74925929=91, srv2139978042=60, srv1624426746=33, srv1800694279=44, srv437217299=73, srv171445303=38, srv701080180=88, srv1131609332=5, srv288449217=65, srv1460782924=24, srv1602917888=31, srv1755666932=40, srv165397939=35, srv335428563=69, srv705407629=89, srv615977597=85, srv117002844=13, srv513352721=78, srv879178437=98, srv115580900=8, srv1694181643=37, srv514136320=79, srv848334531=96, srv1520663653=28, srv18547335=48, srv260742302=63, srv1646790675=34, srv274214721=64, srv1996530885=53, srv116979828=12, srv492552268=76, srv2146461733=61, srv789702934=93, srv1095858553=2, srv1293153957=17, srv568887862=81, srv1164155242=9, srv408547582=71, srv560480358=80, srv658839701=86} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:16,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:16,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:16,998 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:16,999 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:17,000 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 12:54:17,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 12:54:17,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 12:54:17,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 12:54:17,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 12:54:17,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 12:54:17,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 12:54:17,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 12:54:17,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 12:54:17,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 12:54:17,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 12:54:17,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 12:54:17,007 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 12:54:17,191 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04075726631474507, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06488702086698189, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04075726631474507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17158333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24633333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:17,191 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 12:54:17,192 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1771376190802645, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04075726631474507, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06488702086698189, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04075726631474507, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17158333333333334, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24633333333333332, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 12:54:17,493 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 458 ms to try 5000 different iterations.  Found a solution that moves 2642 regions; Going from a computed imbalance of 0.1771376190802645 to a new imbalance of 0.01611670911948753. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020931660964362653); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0372180760543205, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29355555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020931660964362653); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:17,505 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv406165961:78 , srv1839642147:79 , srv2085004527:79 , srv491866102:79 , srv100551892:80 , srv1800694279:80 , srv701080180:80 , srv1131609332:81 , srv1326106814:81 , srv1608387864:81 , srv1099353935:82 , srv131508293:82 , srv165397939:82 , srv1770751599:82 , srv1811372937:82 , srv2048184643:82 , srv494849059:82 , srv919680812:82 , srv1095858553:83 , srv116979828:83 , srv171445303:83 , srv1927808531:83 , srv2087352896:83 , srv718344655:83 , srv115580900:84 , srv117002844:84 , srv1459065897:84 , srv1460782924:84 , srv274214721:84 , srv469184640:84 , srv705407629:84 , srv789702934:84 , srv1167220210:85 , srv1289358191:85 , srv1444687027:85 , srv1894224832:85 , srv514136320:85 , srv783264483:85 , srv819851503:85 , srv1097348085:86 , srv1141611690:86 , srv1168993528:86 , srv1180992657:86 , srv1504807184:86 , srv1572573157:86 , srv1996530885:86 , srv2139978042:86 , srv227592999:86 , srv288449217:86 , srv320743146:86 , srv568887862:86 , srv581892405:86 , srv608392041:86 , srv697893439:86 , srv87239816:86 , srv1152097368:87 , srv1528891442:87 , srv1602917888:87 , srv1624426746:87 , srv1755666932:87 , srv1817146903:87 , srv18547335:87 , srv2146461733:87 , srv318137291:87 , srv335428563:87 , srv437217299:87 , srv615977597:87 , srv1181911679:88 , srv1431219816:88 , srv1513571950:88 , srv1520663653:88 , srv175796375:88 , srv292678086:88 , srv408547582:88 , srv492552268:88 , srv560480358:88 , srv74925929:88 , srv848334531:88 , srv879178437:88 , srv1293153957:89 , srv1510412012:89 , srv1670345235:89 , srv1694181643:89 , srv173006868:89 , srv1769351916:89 , srv1919822980:89 , srv2049376735:89 , srv260742302:89 , srv432571802:89 , srv60199277:89 , srv1164155242:90 , srv1913073400:90 , srv513352721:90 , srv658839701:90 , srv817755986:90 , srv1025425038:91 , srv1646790675:91 , srv2010539834:91 , srv2010968701:95 , srv1372387710:508 }
2023-03-10 12:54:17,509 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:17,510 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1608387864=32, srv1168993528=11, srv1913073400=50, srv491866102=75, srv87239816=97, srv1141611690=6, srv608392041=84, srv175796375=41, srv131508293=18, srv227592999=62, srv1770751599=43, srv1097348085=3, srv2085004527=58, srv2010968701=55, srv1152097368=7, srv1431219816=21, srv1180992657=14, srv1817146903=46, srv1894224832=49, srv2087352896=59, srv783264483=92, srv718344655=90, srv1459065897=23, srv469184640=74, srv2049376735=57, srv494849059=77, srv1528891442=29, srv1513571950=27, srv318137291=67, srv1504807184=25, srv100551892=0, srv2048184643=56, srv1444687027=22, srv1326106814=19, srv581892405=82, srv919680812=99, srv1025425038=1, srv1510412012=26, srv1919822980=51, srv1811372937=45, srv1372387710=20, srv406165961=70, srv1670345235=36, srv1769351916=42, srv320743146=68, srv1181911679=15, srv173006868=39, srv819851503=95, srv432571802=72, srv1167220210=10, srv1289358191=16, srv60199277=83, srv1099353935=4, srv292678086=66, srv697893439=87, srv1572573157=30, srv1927808531=52, srv2010539834=54, srv817755986=94, srv1839642147=47, srv74925929=91, srv2139978042=60, srv1624426746=33, srv1800694279=44, srv437217299=73, srv171445303=38, srv701080180=88, srv1131609332=5, srv288449217=65, srv1460782924=24, srv1602917888=31, srv1755666932=40, srv165397939=35, srv335428563=69, srv705407629=89, srv615977597=85, srv117002844=13, srv513352721=78, srv879178437=98, srv115580900=8, srv1694181643=37, srv514136320=79, srv848334531=96, srv1520663653=28, srv18547335=48, srv260742302=63, srv1646790675=34, srv274214721=64, srv1996530885=53, srv116979828=12, srv492552268=76, srv2146461733=61, srv789702934=93, srv1095858553=2, srv1293153957=17, srv568887862=81, srv1164155242=9, srv408547582=71, srv560480358=80, srv658839701=86} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:17,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:17,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:17,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:17,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:17,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:17,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:17,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 12:54:17,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 12:54:17,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 12:54:17,527 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 12:54:17,528 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 12:54:17,529 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 12:54:17,639 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020931660964362653); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0372180760543205, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020931660964362653); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:17,639 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 12:54:17,639 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016098209440085956, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.020931660964362653); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0372180760543205, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.020931660964362653); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.01275); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04833333333333333, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 12:54:17,901 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 362 ms to try 5000 different iterations.  Found a solution that moves 1027 regions; Going from a computed imbalance of 0.016098209440085956 to a new imbalance of 8.994533925264855E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006559178363224968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009534625892455923); MoveCostFunction : (multiplier=7.0, imbalance=0.11411111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006559178363224968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:17,904 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1164155242:82 , srv1572573157:82 , srv2010968701:82 , srv260742302:82 , srv560480358:82 , srv1097348085:83 , srv1152097368:83 , srv1181911679:83 , srv1460782924:83 , srv1025425038:84 , srv1141611690:84 , srv116979828:84 , srv1289358191:84 , srv1459065897:84 , srv1520663653:84 , srv1646790675:84 , srv1670345235:84 , srv1694181643:84 , srv173006868:84 , srv175796375:84 , srv1839642147:84 , srv1919822980:84 , srv227592999:84 , srv491866102:84 , srv608392041:84 , srv658839701:84 , srv848334531:84 , srv879178437:84 , srv1167220210:85 , srv1180992657:85 , srv1326106814:85 , srv1504807184:85 , srv1513571950:85 , srv1528891442:85 , srv1602917888:85 , srv1624426746:85 , srv171445303:85 , srv1769351916:85 , srv1811372937:85 , srv18547335:85 , srv1913073400:85 , srv1996530885:85 , srv2010539834:85 , srv2048184643:85 , srv2049376735:85 , srv2085004527:85 , srv2139978042:85 , srv2146461733:85 , srv288449217:85 , srv320743146:85 , srv432571802:85 , srv469184640:85 , srv513352721:85 , srv60199277:85 , srv615977597:85 , srv697893439:85 , srv701080180:85 , srv718344655:85 , srv817755986:85 , srv1099353935:86 , srv115580900:86 , srv1168993528:86 , srv117002844:86 , srv1293153957:86 , srv1431219816:86 , srv1510412012:86 , srv165397939:86 , srv1770751599:86 , srv1817146903:86 , srv1894224832:86 , srv274214721:86 , srv292678086:86 , srv406165961:86 , srv408547582:86 , srv581892405:86 , srv74925929:86 , srv819851503:86 , srv87239816:86 , srv100551892:87 , srv1131609332:87 , srv131508293:87 , srv1608387864:87 , srv1927808531:87 , srv2087352896:87 , srv318137291:87 , srv335428563:87 , srv437217299:87 , srv492552268:87 , srv494849059:87 , srv568887862:87 , srv783264483:87 , srv1444687027:88 , srv1755666932:88 , srv1800694279:88 , srv514136320:88 , srv705407629:88 , srv919680812:88 , srv1095858553:89 , srv789702934:89 , srv1372387710:556 }
2023-03-10 12:54:17,907 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:17,909 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1608387864=32, srv1168993528=11, srv1913073400=50, srv491866102=75, srv87239816=97, srv1141611690=6, srv608392041=84, srv175796375=41, srv131508293=18, srv227592999=62, srv1770751599=43, srv1097348085=3, srv2085004527=58, srv2010968701=55, srv1152097368=7, srv1431219816=21, srv1180992657=14, srv1817146903=46, srv1894224832=49, srv2087352896=59, srv783264483=92, srv718344655=90, srv1459065897=23, srv469184640=74, srv2049376735=57, srv494849059=77, srv1528891442=29, srv1513571950=27, srv318137291=67, srv1504807184=25, srv100551892=0, srv2048184643=56, srv1444687027=22, srv1326106814=19, srv581892405=82, srv919680812=99, srv1025425038=1, srv1510412012=26, srv1919822980=51, srv1811372937=45, srv1372387710=20, srv406165961=70, srv1670345235=36, srv1769351916=42, srv320743146=68, srv1181911679=15, srv173006868=39, srv819851503=95, srv432571802=72, srv1167220210=10, srv1289358191=16, srv60199277=83, srv1099353935=4, srv292678086=66, srv697893439=87, srv1572573157=30, srv1927808531=52, srv2010539834=54, srv817755986=94, srv1839642147=47, srv74925929=91, srv2139978042=60, srv1624426746=33, srv1800694279=44, srv437217299=73, srv171445303=38, srv701080180=88, srv1131609332=5, srv288449217=65, srv1460782924=24, srv1602917888=31, srv1755666932=40, srv165397939=35, srv335428563=69, srv705407629=89, srv615977597=85, srv117002844=13, srv513352721=78, srv879178437=98, srv115580900=8, srv1694181643=37, srv514136320=79, srv848334531=96, srv1520663653=28, srv18547335=48, srv260742302=63, srv1646790675=34, srv274214721=64, srv1996530885=53, srv116979828=12, srv492552268=76, srv2146461733=61, srv789702934=93, srv1095858553=2, srv1293153957=17, srv568887862=81, srv1164155242=9, srv408547582=71, srv560480358=80, srv658839701=86} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:17,927 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:17,928 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:17,929 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 12:54:17,930 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 12:54:17,931 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 12:54:17,932 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 12:54:17,933 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 12:54:17,934 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 12:54:17,935 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 12:54:17,936 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 12:54:17,937 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 12:54:17,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 12:54:17,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 12:54:17,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 12:54:17,938 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 12:54:18,054 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006559178363224968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009534625892455923); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006559178363224968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:18,054 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 12:54:18,054 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.922621848257214E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006559178363224968); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.009534625892455923); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.006559178363224968); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004083333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 12:54:18,307 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 353 ms to try 5000 different iterations.  Found a solution that moves 290 regions; Going from a computed imbalance of 8.922621848257214E-4 to a new imbalance of 3.622652836131756E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.03222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:18,308 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv789702934:82 , srv1095858553:83 , srv406165961:83 , srv783264483:83 , srv100551892:84 , srv1025425038:84 , srv1099353935:84 , srv1168993528:84 , srv131508293:84 , srv1431219816:84 , srv1444687027:84 , srv1460782924:84 , srv1513571950:84 , srv1528891442:84 , srv1608387864:84 , srv1670345235:84 , srv1755666932:84 , srv1800694279:84 , srv1811372937:84 , srv1913073400:84 , srv1927808531:84 , srv1996530885:84 , srv2010968701:84 , srv2048184643:84 , srv2049376735:84 , srv2087352896:84 , srv318137291:84 , srv320743146:84 , srv408547582:84 , srv437217299:84 , srv494849059:84 , srv513352721:84 , srv514136320:84 , srv560480358:84 , srv568887862:84 , srv581892405:84 , srv608392041:84 , srv705407629:84 , srv817755986:84 , srv819851503:84 , srv87239816:84 , srv919680812:84 , srv1097348085:85 , srv1141611690:85 , srv1152097368:85 , srv115580900:85 , srv1164155242:85 , srv1167220210:85 , srv116979828:85 , srv117002844:85 , srv1180992657:85 , srv1181911679:85 , srv1293153957:85 , srv1326106814:85 , srv1459065897:85 , srv1504807184:85 , srv1510412012:85 , srv1520663653:85 , srv1572573157:85 , srv1602917888:85 , srv1624426746:85 , srv1646790675:85 , srv165397939:85 , srv1694181643:85 , srv171445303:85 , srv173006868:85 , srv175796375:85 , srv1769351916:85 , srv1770751599:85 , srv1817146903:85 , srv1839642147:85 , srv18547335:85 , srv1894224832:85 , srv1919822980:85 , srv2010539834:85 , srv2085004527:85 , srv2139978042:85 , srv227592999:85 , srv260742302:85 , srv274214721:85 , srv288449217:85 , srv292678086:85 , srv335428563:85 , srv469184640:85 , srv491866102:85 , srv492552268:85 , srv60199277:85 , srv615977597:85 , srv658839701:85 , srv697893439:85 , srv701080180:85 , srv74925929:85 , srv848334531:85 , srv879178437:85 , srv1131609332:86 , srv1289358191:86 , srv432571802:86 , srv718344655:86 , srv2146461733:87 , srv1372387710:626 }
2023-03-10 12:54:18,311 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:18,312 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1608387864=32, srv1168993528=11, srv1913073400=50, srv491866102=75, srv87239816=97, srv1141611690=6, srv608392041=84, srv175796375=41, srv131508293=18, srv227592999=62, srv1770751599=43, srv1097348085=3, srv2085004527=58, srv2010968701=55, srv1152097368=7, srv1431219816=21, srv1180992657=14, srv1817146903=46, srv1894224832=49, srv2087352896=59, srv783264483=92, srv718344655=90, srv1459065897=23, srv469184640=74, srv2049376735=57, srv494849059=77, srv1528891442=29, srv1513571950=27, srv318137291=67, srv1504807184=25, srv100551892=0, srv2048184643=56, srv1444687027=22, srv1326106814=19, srv581892405=82, srv919680812=99, srv1025425038=1, srv1510412012=26, srv1919822980=51, srv1811372937=45, srv1372387710=20, srv406165961=70, srv1670345235=36, srv1769351916=42, srv320743146=68, srv1181911679=15, srv173006868=39, srv819851503=95, srv432571802=72, srv1167220210=10, srv1289358191=16, srv60199277=83, srv1099353935=4, srv292678086=66, srv697893439=87, srv1572573157=30, srv1927808531=52, srv2010539834=54, srv817755986=94, srv1839642147=47, srv74925929=91, srv2139978042=60, srv1624426746=33, srv1800694279=44, srv437217299=73, srv171445303=38, srv701080180=88, srv1131609332=5, srv288449217=65, srv1460782924=24, srv1602917888=31, srv1755666932=40, srv165397939=35, srv335428563=69, srv705407629=89, srv615977597=85, srv117002844=13, srv513352721=78, srv879178437=98, srv115580900=8, srv1694181643=37, srv514136320=79, srv848334531=96, srv1520663653=28, srv18547335=48, srv260742302=63, srv1646790675=34, srv274214721=64, srv1996530885=53, srv116979828=12, srv492552268=76, srv2146461733=61, srv789702934=93, srv1095858553=2, srv1293153957=17, srv568887862=81, srv1164155242=9, srv408547582=71, srv560480358=80, srv658839701=86} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:18,322 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:18,323 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 12:54:18,324 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 12:54:18,325 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 12:54:18,326 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 12:54:18,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 12:54:18,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 12:54:18,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 12:54:18,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 12:54:18,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 12:54:18,327 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 12:54:18,328 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 12:54:18,329 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 12:54:18,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 12:54:18,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 12:54:18,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 12:54:18,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 12:54:18,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 12:54:18,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 12:54:18,447 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:18,448 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-03-10 12:54:18,448 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=3.419590495993334E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002118805753879094); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=2.5E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-03-10 12:54:18,649 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 310 ms to try 5000 different iterations.  Found a solution that moves 20 regions; Going from a computed imbalance of 3.419590495993334E-5 to a new imbalance of 9.188161102791466E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0022222222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:18,650 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1095858553:83 , srv1459065897:83 , srv1572573157:83 , srv1624426746:83 , srv2146461733:83 , srv432571802:83 , srv74925929:83 , srv919680812:83 , srv100551892:84 , srv1025425038:84 , srv1097348085:84 , srv1131609332:84 , srv1141611690:84 , srv1152097368:84 , srv115580900:84 , srv1164155242:84 , srv116979828:84 , srv117002844:84 , srv1180992657:84 , srv1181911679:84 , srv1289358191:84 , srv1293153957:84 , srv131508293:84 , srv1431219816:84 , srv1444687027:84 , srv1460782924:84 , srv1504807184:84 , srv1510412012:84 , srv1513571950:84 , srv1520663653:84 , srv1528891442:84 , srv1602917888:84 , srv1608387864:84 , srv1646790675:84 , srv165397939:84 , srv1670345235:84 , srv1694181643:84 , srv171445303:84 , srv173006868:84 , srv1755666932:84 , srv175796375:84 , srv1769351916:84 , srv1770751599:84 , srv1800694279:84 , srv1811372937:84 , srv1817146903:84 , srv1839642147:84 , srv18547335:84 , srv1894224832:84 , srv1919822980:84 , srv1927808531:84 , srv1996530885:84 , srv2010539834:84 , srv2010968701:84 , srv2049376735:84 , srv2085004527:84 , srv2087352896:84 , srv2139978042:84 , srv227592999:84 , srv260742302:84 , srv274214721:84 , srv288449217:84 , srv292678086:84 , srv318137291:84 , srv335428563:84 , srv408547582:84 , srv437217299:84 , srv469184640:84 , srv491866102:84 , srv492552268:84 , srv494849059:84 , srv513352721:84 , srv514136320:84 , srv560480358:84 , srv568887862:84 , srv581892405:84 , srv60199277:84 , srv608392041:84 , srv615977597:84 , srv658839701:84 , srv697893439:84 , srv701080180:84 , srv705407629:84 , srv718344655:84 , srv783264483:84 , srv789702934:84 , srv817755986:84 , srv819851503:84 , srv848334531:84 , srv87239816:84 , srv879178437:84 , srv1099353935:85 , srv1167220210:85 , srv1168993528:85 , srv1326106814:85 , srv1913073400:85 , srv2048184643:85 , srv320743146:85 , srv406165961:85 , srv1372387710:684 }
2023-03-10 12:54:18,653 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:18,654 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1608387864=32, srv1168993528=11, srv1913073400=50, srv491866102=75, srv87239816=97, srv1141611690=6, srv608392041=84, srv175796375=41, srv131508293=18, srv227592999=62, srv1770751599=43, srv1097348085=3, srv2085004527=58, srv2010968701=55, srv1152097368=7, srv1431219816=21, srv1180992657=14, srv1817146903=46, srv1894224832=49, srv2087352896=59, srv783264483=92, srv718344655=90, srv1459065897=23, srv469184640=74, srv2049376735=57, srv494849059=77, srv1528891442=29, srv1513571950=27, srv318137291=67, srv1504807184=25, srv100551892=0, srv2048184643=56, srv1444687027=22, srv1326106814=19, srv581892405=82, srv919680812=99, srv1025425038=1, srv1510412012=26, srv1919822980=51, srv1811372937=45, srv1372387710=20, srv406165961=70, srv1670345235=36, srv1769351916=42, srv320743146=68, srv1181911679=15, srv173006868=39, srv819851503=95, srv432571802=72, srv1167220210=10, srv1289358191=16, srv60199277=83, srv1099353935=4, srv292678086=66, srv697893439=87, srv1572573157=30, srv1927808531=52, srv2010539834=54, srv817755986=94, srv1839642147=47, srv74925929=91, srv2139978042=60, srv1624426746=33, srv1800694279=44, srv437217299=73, srv171445303=38, srv701080180=88, srv1131609332=5, srv288449217=65, srv1460782924=24, srv1602917888=31, srv1755666932=40, srv165397939=35, srv335428563=69, srv705407629=89, srv615977597=85, srv117002844=13, srv513352721=78, srv879178437=98, srv115580900=8, srv1694181643=37, srv514136320=79, srv848334531=96, srv1520663653=28, srv18547335=48, srv260742302=63, srv1646790675=34, srv274214721=64, srv1996530885=53, srv116979828=12, srv492552268=76, srv2146461733=61, srv789702934=93, srv1095858553=2, srv1293153957=17, srv568887862=81, srv1164155242=9, srv408547582=71, srv560480358=80, srv658839701=86} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:18,664 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-03-10 12:54:18,665 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-03-10 12:54:18,666 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-03-10 12:54:18,667 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-03-10 12:54:18,668 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-03-10 12:54:18,669 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-03-10 12:54:18,781 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=9.048118109592553E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.002010075630518424); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:18,781 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1095858553:83 , srv1459065897:83 , srv1572573157:83 , srv1624426746:83 , srv2146461733:83 , srv432571802:83 , srv74925929:83 , srv919680812:83 , srv100551892:84 , srv1025425038:84 , srv1097348085:84 , srv1131609332:84 , srv1141611690:84 , srv1152097368:84 , srv115580900:84 , srv1164155242:84 , srv116979828:84 , srv117002844:84 , srv1180992657:84 , srv1181911679:84 , srv1289358191:84 , srv1293153957:84 , srv131508293:84 , srv1431219816:84 , srv1444687027:84 , srv1460782924:84 , srv1504807184:84 , srv1510412012:84 , srv1513571950:84 , srv1520663653:84 , srv1528891442:84 , srv1602917888:84 , srv1608387864:84 , srv1646790675:84 , srv165397939:84 , srv1670345235:84 , srv1694181643:84 , srv171445303:84 , srv173006868:84 , srv1755666932:84 , srv175796375:84 , srv1769351916:84 , srv1770751599:84 , srv1800694279:84 , srv1811372937:84 , srv1817146903:84 , srv1839642147:84 , srv18547335:84 , srv1894224832:84 , srv1919822980:84 , srv1927808531:84 , srv1996530885:84 , srv2010539834:84 , srv2010968701:84 , srv2049376735:84 , srv2085004527:84 , srv2087352896:84 , srv2139978042:84 , srv227592999:84 , srv260742302:84 , srv274214721:84 , srv288449217:84 , srv292678086:84 , srv318137291:84 , srv335428563:84 , srv408547582:84 , srv437217299:84 , srv469184640:84 , srv491866102:84 , srv492552268:84 , srv494849059:84 , srv513352721:84 , srv514136320:84 , srv560480358:84 , srv568887862:84 , srv581892405:84 , srv60199277:84 , srv608392041:84 , srv615977597:84 , srv658839701:84 , srv697893439:84 , srv701080180:84 , srv705407629:84 , srv718344655:84 , srv783264483:84 , srv789702934:84 , srv817755986:84 , srv819851503:84 , srv848334531:84 , srv87239816:84 , srv879178437:84 , srv1099353935:85 , srv1167220210:85 , srv1168993528:85 , srv1326106814:85 , srv1913073400:85 , srv2048184643:85 , srv320743146:85 , srv406165961:85 , srv1372387710:684 }
2023-03-10 12:54:18,807 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=281 (was 271) - SystemLoadAverage LEAK? -, ProcessCount=170 (was 170), AvailableMemoryMB=5500 (was 6032)
2023-03-10 12:54:18,815 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=281, ProcessCount=170, AvailableMemoryMB=5498
2023-03-10 12:54:18,816 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-03-10 12:54:18,816 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-03-10 12:54:18,825 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1004623844:84 , srv1015275869:84 , srv1187621651:84 , srv1200899977:84 , srv1248389915:84 , srv1364762474:84 , srv1434710117:84 , srv14847679:84 , srv1604333102:84 , srv1604874478:84 , srv16709690:84 , srv1706285417:84 , srv1709161940:84 , srv1775764914:84 , srv1928974473:84 , srv194571144:84 , srv199982428:84 , srv348658223:84 , srv425661445:84 , srv427602481:84 , srv439639170:84 , srv445158379:84 , srv482583795:84 , srv509610761:84 , srv61911222:84 , srv758213173:84 , srv886196146:84 , srv915501798:84 , srv958018533:84 , srv1739784038:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-03-10 12:54:18,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:54:18,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706285417=11, srv425661445=19, srv482583795=23, srv14847679=7, srv1604874478=9, srv915501798=28, srv1739784038=13, srv199982428=17, srv958018533=29, srv1364762474=5, srv16709690=10, srv1775764914=14, srv1248389915=4, srv1015275869=1, srv1709161940=12, srv439639170=21, srv1200899977=3, srv1434710117=6, srv886196146=27, srv1604333102=8, srv1187621651=2, srv427602481=20, srv445158379=22, srv509610761=24, srv61911222=25, srv758213173=26, srv1928974473=15, srv194571144=16, srv1004623844=0, srv348658223=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:54:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:54:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:54:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:54:18,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:54:18,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:54:18,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:54:18,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:54:18,838 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-10 12:54:18,849 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1153006041278783, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:54:18,849 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912556351702976, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1153006041278783, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-03-10 12:56:18,841 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6754631 different iterations.  Found a solution that moves 1849 regions; Going from a computed imbalance of 0.9912556351702976 to a new imbalance of 4.3156582404131396E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6848148148148148, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:56:18,847 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1004623844:90 , srv1015275869:90 , srv1187621651:90 , srv1200899977:90 , srv1248389915:90 , srv1364762474:90 , srv1434710117:90 , srv14847679:90 , srv1604333102:90 , srv1604874478:90 , srv16709690:90 , srv1706285417:90 , srv1709161940:90 , srv1739784038:90 , srv1775764914:90 , srv1928974473:90 , srv194571144:90 , srv199982428:90 , srv348658223:90 , srv425661445:90 , srv427602481:90 , srv439639170:90 , srv445158379:90 , srv482583795:90 , srv509610761:90 , srv61911222:90 , srv758213173:90 , srv886196146:90 , srv915501798:90 , srv958018533:90 }
2023-03-10 12:56:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-03-10 12:56:18,848 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1706285417=11, srv425661445=19, srv482583795=23, srv14847679=7, srv1604874478=9, srv915501798=28, srv1739784038=13, srv199982428=17, srv958018533=29, srv1364762474=5, srv16709690=10, srv1775764914=14, srv1248389915=4, srv1015275869=1, srv1709161940=12, srv439639170=21, srv1200899977=3, srv1434710117=6, srv886196146=27, srv1604333102=8, srv1187621651=2, srv427602481=20, srv445158379=22, srv509610761=24, srv61911222=25, srv758213173=26, srv1928974473=15, srv194571144=16, srv1004623844=0, srv348658223=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-03-10 12:56:18,852 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-03-10 12:56:18,853 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-03-10 12:56:18,865 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-03-10 12:56:18,865 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1004623844:90 , srv1015275869:90 , srv1187621651:90 , srv1200899977:90 , srv1248389915:90 , srv1364762474:90 , srv1434710117:90 , srv14847679:90 , srv1604333102:90 , srv1604874478:90 , srv16709690:90 , srv1706285417:90 , srv1709161940:90 , srv1739784038:90 , srv1775764914:90 , srv1928974473:90 , srv194571144:90 , srv199982428:90 , srv348658223:90 , srv425661445:90 , srv427602481:90 , srv439639170:90 , srv445158379:90 , srv482583795:90 , srv509610761:90 , srv61911222:90 , srv758213173:90 , srv886196146:90 , srv915501798:90 , srv958018533:90 }
2023-03-10 12:56:18,880 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=190 (was 281), ProcessCount=170 (was 170), AvailableMemoryMB=5191 (was 5498)