2023-11-16 03:55:17,269 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-11-16 03:55:17,538 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-11-16 03:55:17,650 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-11-16 03:55:17,720 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-16 03:55:17,730 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-16 03:55:17,745 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=170, AvailableMemoryMB=5028
2023-11-16 03:55:17,746 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-16 03:55:17,747 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-11-16 03:55:17,877 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1017270171:84 , srv1097472324:84 , srv1111073185:84 , srv1131591443:84 , srv114720162:84 , srv1153864153:84 , srv1157589761:84 , srv1168629486:84 , srv1199535853:84 , srv1219154223:84 , srv1258281299:84 , srv1275418408:84 , srv1285205353:84 , srv1299357302:84 , srv1301316565:84 , srv1308966345:84 , srv1315615445:84 , srv1360348456:84 , srv1365382056:84 , srv1391980587:84 , srv1400469239:84 , srv1405373269:84 , srv1450941121:84 , srv1479872889:84 , srv1541577908:84 , srv1565885030:84 , srv158151449:84 , srv1606275020:84 , srv1618989724:84 , srv1631081464:84 , srv1651741792:84 , srv165395571:84 , srv1670492614:84 , srv1673303422:84 , srv1687534418:84 , srv1715493962:84 , srv1715633435:84 , srv1745317707:84 , srv1766129494:84 , srv1770443827:84 , srv1771210747:84 , srv177813736:84 , srv1783464740:84 , srv1798421398:84 , srv1846851722:84 , srv1852948133:84 , srv1857308619:84 , srv1909150514:84 , srv1922890433:84 , srv1928423964:84 , srv1956433424:84 , srv1982531417:84 , srv199509390:84 , srv2041481489:84 , srv2072755643:84 , srv2123505604:84 , srv2146436661:84 , srv230856210:84 , srv242809047:84 , srv303820221:84 , srv318653403:84 , srv339066005:84 , srv360147107:84 , srv375101733:84 , srv396658469:84 , srv402516341:84 , srv410788949:84 , srv414745456:84 , srv416980128:84 , srv422409462:84 , srv454436883:84 , srv47043225:84 , srv482026286:84 , srv486440841:84 , srv512497620:84 , srv568694476:84 , srv590852417:84 , srv60558672:84 , srv606699751:84 , srv616723546:84 , srv628269580:84 , srv642042101:84 , srv653964680:84 , srv685002252:84 , srv704190946:84 , srv753011170:84 , srv756367854:84 , srv786734974:84 , srv814572518:84 , srv857329039:84 , srv863227998:84 , srv877956434:84 , srv889769592:84 , srv900424713:84 , srv904401792:84 , srv926041777:84 , srv968580510:84 , srv994293830:84 , srv997281223:84 , srv1197998020:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-11-16 03:55:17,886 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:17,892 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1770443827=40, srv1308966345=16, srv360147107=63, srv303820221=60, srv1618989724=29, srv616723546=80, srv2072755643=55, srv653964680=83, srv410788949=67, srv1017270171=0, srv889769592=93, srv422409462=70, srv1219154223=10, srv199509390=53, srv877956434=92, srv47043225=72, srv900424713=94, srv482026286=73, srv242809047=59, srv2146436661=57, srv60558672=78, srv606699751=79, srv1405373269=22, srv1745317707=38, srv1982531417=52, srv1857308619=47, srv1798421398=44, srv1285205353=13, srv685002252=84, srv486440841=74, srv1258281299=11, srv1651741792=31, srv904401792=95, srv512497620=75, srv416980128=69, srv786734974=88, srv2041481489=54, srv756367854=87, srv968580510=97, srv1391980587=20, srv1111073185=2, srv230856210=58, srv1365382056=19, srv1909150514=48, srv994293830=98, srv165395571=32, srv339066005=62, srv1631081464=30, srv1097472324=1, srv1670492614=33, srv1846851722=45, srv396658469=65, srv1565885030=26, srv1687534418=35, srv1400469239=21, srv628269580=81, srv704190946=85, srv1673303422=34, srv1131591443=3, srv1301316565=15, srv1766129494=39, srv375101733=64, srv318653403=61, srv1360348456=18, srv1479872889=24, srv2123505604=56, srv857329039=90, srv814572518=89, srv1715633435=37, srv158151449=27, srv1783464740=43, srv1168629486=7, srv1157589761=6, srv414745456=68, srv454436883=71, srv590852417=77, srv926041777=96, srv1275418408=12, srv1852948133=46, srv753011170=86, srv1606275020=28, srv1956433424=51, srv1199535853=9, srv1928423964=50, srv642042101=82, srv863227998=91, srv1315615445=17, srv1541577908=25, srv568694476=76, srv402516341=66, srv1153864153=5, srv1922890433=49, srv1197998020=8, srv1771210747=41, srv114720162=4, srv177813736=42, srv1450941121=23, srv1715493962=36, srv1299357302=14, srv997281223=99} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:17,967 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:17,968 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:17,969 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-16 03:55:17,970 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-16 03:55:17,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-16 03:55:17,972 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-16 03:55:17,973 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-16 03:55:17,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-16 03:55:17,975 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-16 03:55:17,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-16 03:55:17,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-16 03:55:17,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-16 03:55:17,976 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-16 03:55:18,144 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:18,145 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-16 03:55:18,145 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-16 03:55:18,480 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 484 ms to try 5000 different iterations.  Found a solution that moves 4709 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17470850127868037. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04309974742337359, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058750552209836444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5232222222222223, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04309974742337359, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1695, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:18,502 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv230856210:80 , srv1541577908:81 , srv416980128:81 , srv1157589761:82 , srv1405373269:82 , srv1479872889:82 , srv1715633435:82 , srv303820221:82 , srv814572518:82 , srv114720162:83 , srv1301316565:83 , srv1715493962:83 , srv1745317707:83 , srv402516341:83 , srv482026286:83 , srv590852417:83 , srv60558672:83 , srv606699751:83 , srv753011170:83 , srv968580510:83 , srv1153864153:84 , srv1219154223:84 , srv1299357302:84 , srv1631081464:84 , srv1766129494:84 , srv1798421398:84 , srv1922890433:84 , srv396658469:84 , srv653964680:84 , srv857329039:84 , srv863227998:84 , srv889769592:84 , srv1111073185:85 , srv1360348456:85 , srv1400469239:85 , srv177813736:85 , srv568694476:85 , srv628269580:85 , srv786734974:85 , srv877956434:85 , srv900424713:85 , srv1168629486:86 , srv1275418408:86 , srv1565885030:86 , srv1606275020:86 , srv1687534418:86 , srv1783464740:86 , srv2041481489:86 , srv2072755643:86 , srv339066005:86 , srv454436883:86 , srv512497620:86 , srv642042101:86 , srv926041777:86 , srv994293830:86 , srv1097472324:87 , srv1258281299:87 , srv1365382056:87 , srv1771210747:87 , srv1852948133:87 , srv1857308619:87 , srv1909150514:87 , srv1956433424:87 , srv375101733:87 , srv47043225:87 , srv616723546:87 , srv904401792:87 , srv1131591443:88 , srv1450941121:88 , srv1618989724:88 , srv1651741792:88 , srv165395571:88 , srv1673303422:88 , srv1770443827:88 , srv1928423964:88 , srv199509390:88 , srv2123505604:88 , srv318653403:88 , srv414745456:88 , srv1017270171:89 , srv1285205353:89 , srv1315615445:89 , srv1391980587:89 , srv2146436661:89 , srv360147107:89 , srv410788949:89 , srv422409462:89 , srv486440841:89 , srv704190946:89 , srv1199535853:90 , srv158151449:90 , srv1670492614:90 , srv1846851722:90 , srv685002252:90 , srv1982531417:91 , srv997281223:92 , srv1308966345:93 , srv242809047:93 , srv756367854:94 , srv1197998020:473 }
2023-11-16 03:55:18,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:18,506 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1770443827=40, srv1308966345=16, srv360147107=63, srv303820221=60, srv1618989724=29, srv616723546=80, srv2072755643=55, srv653964680=83, srv410788949=67, srv1017270171=0, srv889769592=93, srv422409462=70, srv1219154223=10, srv199509390=53, srv877956434=92, srv47043225=72, srv900424713=94, srv482026286=73, srv242809047=59, srv2146436661=57, srv60558672=78, srv606699751=79, srv1405373269=22, srv1745317707=38, srv1982531417=52, srv1857308619=47, srv1798421398=44, srv1285205353=13, srv685002252=84, srv486440841=74, srv1258281299=11, srv1651741792=31, srv904401792=95, srv512497620=75, srv416980128=69, srv786734974=88, srv2041481489=54, srv756367854=87, srv968580510=97, srv1391980587=20, srv1111073185=2, srv230856210=58, srv1365382056=19, srv1909150514=48, srv994293830=98, srv165395571=32, srv339066005=62, srv1631081464=30, srv1097472324=1, srv1670492614=33, srv1846851722=45, srv396658469=65, srv1565885030=26, srv1687534418=35, srv1400469239=21, srv628269580=81, srv704190946=85, srv1673303422=34, srv1131591443=3, srv1301316565=15, srv1766129494=39, srv375101733=64, srv318653403=61, srv1360348456=18, srv1479872889=24, srv2123505604=56, srv857329039=90, srv814572518=89, srv1715633435=37, srv158151449=27, srv1783464740=43, srv1168629486=7, srv1157589761=6, srv414745456=68, srv454436883=71, srv590852417=77, srv926041777=96, srv1275418408=12, srv1852948133=46, srv753011170=86, srv1606275020=28, srv1956433424=51, srv1199535853=9, srv1928423964=50, srv642042101=82, srv863227998=91, srv1315615445=17, srv1541577908=25, srv568694476=76, srv402516341=66, srv1153864153=5, srv1922890433=49, srv1197998020=8, srv1771210747=41, srv114720162=4, srv177813736=42, srv1450941121=23, srv1715493962=36, srv1299357302=14, srv997281223=99} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:18,519 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:18,520 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:18,521 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-16 03:55:18,522 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-16 03:55:18,523 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-16 03:55:18,524 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-16 03:55:18,525 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-16 03:55:18,526 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-16 03:55:18,658 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04309974742337359, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058750552209836444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04309974742337359, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1695, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:18,658 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-16 03:55:18,658 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17467552815593168, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04309974742337359, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.058750552209836444, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04309974742337359, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1695, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-16 03:55:18,889 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 339 ms to try 5000 different iterations.  Found a solution that moves 2633 regions; Going from a computed imbalance of 0.17467552815593168 to a new imbalance of 0.013885316868405329. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023867803186118492); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03671102306345985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.29255555555555557, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023867803186118492); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:18,902 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv756367854:78 , srv1285205353:79 , srv1308966345:80 , srv1982531417:80 , srv1391980587:81 , srv1618989724:81 , srv199509390:81 , srv486440841:81 , srv997281223:81 , srv1097472324:82 , srv1365382056:82 , srv1450941121:82 , srv1673303422:82 , srv1846851722:82 , srv242809047:82 , srv410788949:82 , srv414745456:82 , srv685002252:82 , srv1017270171:83 , srv1131591443:83 , srv1199535853:83 , srv1315615445:83 , srv158151449:83 , srv1928423964:83 , srv422409462:83 , srv47043225:83 , srv704190946:83 , srv1400469239:84 , srv1770443827:84 , srv1783464740:84 , srv2146436661:84 , srv360147107:84 , srv375101733:84 , srv454436883:84 , srv904401792:84 , srv1111073185:85 , srv1168629486:85 , srv1299357302:85 , srv1606275020:85 , srv165395571:85 , srv1687534418:85 , srv1956433424:85 , srv318653403:85 , srv339066005:85 , srv628269580:85 , srv994293830:85 , srv1565885030:86 , srv1651741792:86 , srv1771210747:86 , srv1857308619:86 , srv1909150514:86 , srv2072755643:86 , srv2123505604:86 , srv568694476:86 , srv616723546:86 , srv653964680:86 , srv877956434:86 , srv900424713:86 , srv114720162:87 , srv1219154223:87 , srv1258281299:87 , srv1275418408:87 , srv1631081464:87 , srv1670492614:87 , srv1745317707:87 , srv177813736:87 , srv1798421398:87 , srv1852948133:87 , srv396658469:87 , srv60558672:87 , srv642042101:87 , srv1360348456:88 , srv1766129494:88 , srv402516341:88 , srv590852417:88 , srv889769592:88 , srv1157589761:89 , srv1301316565:89 , srv1405373269:89 , srv1479872889:89 , srv1715493962:89 , srv1922890433:89 , srv2041481489:89 , srv512497620:89 , srv753011170:89 , srv857329039:89 , srv926041777:89 , srv1153864153:90 , srv1715633435:90 , srv606699751:90 , srv786734974:90 , srv968580510:90 , srv230856210:91 , srv303820221:91 , srv416980128:91 , srv482026286:91 , srv863227998:91 , srv1541577908:93 , srv814572518:93 , srv1197998020:513 }
2023-11-16 03:55:18,905 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:18,906 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1770443827=40, srv1308966345=16, srv360147107=63, srv303820221=60, srv1618989724=29, srv616723546=80, srv2072755643=55, srv653964680=83, srv410788949=67, srv1017270171=0, srv889769592=93, srv422409462=70, srv1219154223=10, srv199509390=53, srv877956434=92, srv47043225=72, srv900424713=94, srv482026286=73, srv242809047=59, srv2146436661=57, srv60558672=78, srv606699751=79, srv1405373269=22, srv1745317707=38, srv1982531417=52, srv1857308619=47, srv1798421398=44, srv1285205353=13, srv685002252=84, srv486440841=74, srv1258281299=11, srv1651741792=31, srv904401792=95, srv512497620=75, srv416980128=69, srv786734974=88, srv2041481489=54, srv756367854=87, srv968580510=97, srv1391980587=20, srv1111073185=2, srv230856210=58, srv1365382056=19, srv1909150514=48, srv994293830=98, srv165395571=32, srv339066005=62, srv1631081464=30, srv1097472324=1, srv1670492614=33, srv1846851722=45, srv396658469=65, srv1565885030=26, srv1687534418=35, srv1400469239=21, srv628269580=81, srv704190946=85, srv1673303422=34, srv1131591443=3, srv1301316565=15, srv1766129494=39, srv375101733=64, srv318653403=61, srv1360348456=18, srv1479872889=24, srv2123505604=56, srv857329039=90, srv814572518=89, srv1715633435=37, srv158151449=27, srv1783464740=43, srv1168629486=7, srv1157589761=6, srv414745456=68, srv454436883=71, srv590852417=77, srv926041777=96, srv1275418408=12, srv1852948133=46, srv753011170=86, srv1606275020=28, srv1956433424=51, srv1199535853=9, srv1928423964=50, srv642042101=82, srv863227998=91, srv1315615445=17, srv1541577908=25, srv568694476=76, srv402516341=66, srv1153864153=5, srv1922890433=49, srv1197998020=8, srv1771210747=41, srv114720162=4, srv177813736=42, srv1450941121=23, srv1715493962=36, srv1299357302=14, srv997281223=99} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:18,919 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:18,920 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-16 03:55:18,921 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-16 03:55:18,922 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-16 03:55:18,923 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-16 03:55:18,924 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-16 03:55:18,925 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-16 03:55:19,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023867803186118492); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03671102306345985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023867803186118492); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:19,041 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-16 03:55:19,041 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.013866880208350692, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023867803186118492); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03671102306345985, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023867803186118492); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.010416666666666666); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04675, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-16 03:55:19,261 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 321 ms to try 5000 different iterations.  Found a solution that moves 972 regions; Going from a computed imbalance of 0.013866880208350692 to a new imbalance of 6.010296670905806E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007672040602626861); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.108, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007672040602626861); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:19,263 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1670492614:81 , srv2041481489:82 , srv242809047:82 , srv512497620:82 , srv863227998:82 , srv926041777:82 , srv786734974:83 , srv814572518:83 , srv1153864153:84 , srv1199535853:84 , srv1258281299:84 , srv1301316565:84 , srv1308966345:84 , srv1360348456:84 , srv1541577908:84 , srv1651741792:84 , srv1771210747:84 , srv1857308619:84 , srv1909150514:84 , srv2123505604:84 , srv482026286:84 , srv606699751:84 , srv616723546:84 , srv756367854:84 , srv968580510:84 , srv997281223:84 , srv1017270171:85 , srv1275418408:85 , srv1565885030:85 , srv158151449:85 , srv165395571:85 , srv1715493962:85 , srv1715633435:85 , srv1766129494:85 , srv1846851722:85 , srv1852948133:85 , srv1922890433:85 , srv1956433424:85 , srv2072755643:85 , srv303820221:85 , srv318653403:85 , srv360147107:85 , srv410788949:85 , srv416980128:85 , srv568694476:85 , srv642042101:85 , srv753011170:85 , srv857329039:85 , srv904401792:85 , srv1131591443:86 , srv1168629486:86 , srv1219154223:86 , srv1315615445:86 , srv1606275020:86 , srv1687534418:86 , srv1770443827:86 , srv177813736:86 , srv1798421398:86 , srv1928423964:86 , srv1982531417:86 , srv2146436661:86 , srv339066005:86 , srv375101733:86 , srv396658469:86 , srv422409462:86 , srv590852417:86 , srv653964680:86 , srv685002252:86 , srv704190946:86 , srv877956434:86 , srv900424713:86 , srv994293830:86 , srv1111073185:87 , srv1157589761:87 , srv1365382056:87 , srv1391980587:87 , srv1400469239:87 , srv1405373269:87 , srv1479872889:87 , srv1631081464:87 , srv1745317707:87 , srv1783464740:87 , srv230856210:87 , srv402516341:87 , srv414745456:87 , srv454436883:87 , srv47043225:87 , srv486440841:87 , srv60558672:87 , srv628269580:87 , srv1097472324:88 , srv114720162:88 , srv1285205353:88 , srv1299357302:88 , srv1450941121:88 , srv1618989724:88 , srv1673303422:88 , srv199509390:88 , srv889769592:88 , srv1197998020:540 }
2023-11-16 03:55:19,267 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:19,269 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1770443827=40, srv1308966345=16, srv360147107=63, srv303820221=60, srv1618989724=29, srv616723546=80, srv2072755643=55, srv653964680=83, srv410788949=67, srv1017270171=0, srv889769592=93, srv422409462=70, srv1219154223=10, srv199509390=53, srv877956434=92, srv47043225=72, srv900424713=94, srv482026286=73, srv242809047=59, srv2146436661=57, srv60558672=78, srv606699751=79, srv1405373269=22, srv1745317707=38, srv1982531417=52, srv1857308619=47, srv1798421398=44, srv1285205353=13, srv685002252=84, srv486440841=74, srv1258281299=11, srv1651741792=31, srv904401792=95, srv512497620=75, srv416980128=69, srv786734974=88, srv2041481489=54, srv756367854=87, srv968580510=97, srv1391980587=20, srv1111073185=2, srv230856210=58, srv1365382056=19, srv1909150514=48, srv994293830=98, srv165395571=32, srv339066005=62, srv1631081464=30, srv1097472324=1, srv1670492614=33, srv1846851722=45, srv396658469=65, srv1565885030=26, srv1687534418=35, srv1400469239=21, srv628269580=81, srv704190946=85, srv1673303422=34, srv1131591443=3, srv1301316565=15, srv1766129494=39, srv375101733=64, srv318653403=61, srv1360348456=18, srv1479872889=24, srv2123505604=56, srv857329039=90, srv814572518=89, srv1715633435=37, srv158151449=27, srv1783464740=43, srv1168629486=7, srv1157589761=6, srv414745456=68, srv454436883=71, srv590852417=77, srv926041777=96, srv1275418408=12, srv1852948133=46, srv753011170=86, srv1606275020=28, srv1956433424=51, srv1199535853=9, srv1928423964=50, srv642042101=82, srv863227998=91, srv1315615445=17, srv1541577908=25, srv568694476=76, srv402516341=66, srv1153864153=5, srv1922890433=49, srv1197998020=8, srv1771210747=41, srv114720162=4, srv177813736=42, srv1450941121=23, srv1715493962=36, srv1299357302=14, srv997281223=99} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:19,290 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:19,291 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:19,292 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:19,293 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-16 03:55:19,294 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-16 03:55:19,295 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-16 03:55:19,296 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-16 03:55:19,297 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-16 03:55:19,298 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-16 03:55:19,299 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-16 03:55:19,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-16 03:55:19,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-16 03:55:19,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-16 03:55:19,300 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-16 03:55:19,416 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007672040602626861); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007672040602626861); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:19,416 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-16 03:55:19,416 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.942235776211134E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.007672040602626861); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.008800061218030102); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.007672040602626861); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=8.333333333333333E-5); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.004916666666666666); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-16 03:55:19,750 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 441 ms to try 5000 different iterations.  Found a solution that moves 271 regions; Going from a computed imbalance of 5.942235776211134E-4 to a new imbalance of 2.7584277952210458E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.030111111111111113, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:19,751 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv889769592:82 , srv1275418408:83 , srv1852948133:83 , srv230856210:83 , srv685002252:83 , srv114720162:84 , srv1153864153:84 , srv1157589761:84 , srv1315615445:84 , srv1391980587:84 , srv1405373269:84 , srv1450941121:84 , srv1479872889:84 , srv1541577908:84 , srv158151449:84 , srv1631081464:84 , srv165395571:84 , srv1673303422:84 , srv1770443827:84 , srv177813736:84 , srv1783464740:84 , srv1922890433:84 , srv2123505604:84 , srv2146436661:84 , srv303820221:84 , srv318653403:84 , srv360147107:84 , srv402516341:84 , srv414745456:84 , srv422409462:84 , srv482026286:84 , srv606699751:84 , srv642042101:84 , srv814572518:84 , srv857329039:84 , srv997281223:84 , srv1017270171:85 , srv1097472324:85 , srv1111073185:85 , srv1131591443:85 , srv1168629486:85 , srv1219154223:85 , srv1258281299:85 , srv1299357302:85 , srv1308966345:85 , srv1360348456:85 , srv1565885030:85 , srv1606275020:85 , srv1618989724:85 , srv1651741792:85 , srv1670492614:85 , srv1687534418:85 , srv1715493962:85 , srv1715633435:85 , srv1745317707:85 , srv1766129494:85 , srv1771210747:85 , srv1798421398:85 , srv1846851722:85 , srv1909150514:85 , srv1928423964:85 , srv1956433424:85 , srv1982531417:85 , srv199509390:85 , srv2041481489:85 , srv2072755643:85 , srv242809047:85 , srv339066005:85 , srv375101733:85 , srv396658469:85 , srv416980128:85 , srv454436883:85 , srv47043225:85 , srv486440841:85 , srv512497620:85 , srv590852417:85 , srv60558672:85 , srv616723546:85 , srv628269580:85 , srv704190946:85 , srv753011170:85 , srv786734974:85 , srv863227998:85 , srv877956434:85 , srv900424713:85 , srv926041777:85 , srv968580510:85 , srv994293830:85 , srv1199535853:86 , srv1285205353:86 , srv1301316565:86 , srv1365382056:86 , srv1400469239:86 , srv1857308619:86 , srv410788949:86 , srv568694476:86 , srv653964680:86 , srv756367854:86 , srv904401792:86 , srv1197998020:616 }
2023-11-16 03:55:19,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:19,755 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1770443827=40, srv1308966345=16, srv360147107=63, srv303820221=60, srv1618989724=29, srv616723546=80, srv2072755643=55, srv653964680=83, srv410788949=67, srv1017270171=0, srv889769592=93, srv422409462=70, srv1219154223=10, srv199509390=53, srv877956434=92, srv47043225=72, srv900424713=94, srv482026286=73, srv242809047=59, srv2146436661=57, srv60558672=78, srv606699751=79, srv1405373269=22, srv1745317707=38, srv1982531417=52, srv1857308619=47, srv1798421398=44, srv1285205353=13, srv685002252=84, srv486440841=74, srv1258281299=11, srv1651741792=31, srv904401792=95, srv512497620=75, srv416980128=69, srv786734974=88, srv2041481489=54, srv756367854=87, srv968580510=97, srv1391980587=20, srv1111073185=2, srv230856210=58, srv1365382056=19, srv1909150514=48, srv994293830=98, srv165395571=32, srv339066005=62, srv1631081464=30, srv1097472324=1, srv1670492614=33, srv1846851722=45, srv396658469=65, srv1565885030=26, srv1687534418=35, srv1400469239=21, srv628269580=81, srv704190946=85, srv1673303422=34, srv1131591443=3, srv1301316565=15, srv1766129494=39, srv375101733=64, srv318653403=61, srv1360348456=18, srv1479872889=24, srv2123505604=56, srv857329039=90, srv814572518=89, srv1715633435=37, srv158151449=27, srv1783464740=43, srv1168629486=7, srv1157589761=6, srv414745456=68, srv454436883=71, srv590852417=77, srv926041777=96, srv1275418408=12, srv1852948133=46, srv753011170=86, srv1606275020=28, srv1956433424=51, srv1199535853=9, srv1928423964=50, srv642042101=82, srv863227998=91, srv1315615445=17, srv1541577908=25, srv568694476=76, srv402516341=66, srv1153864153=5, srv1922890433=49, srv1197998020=8, srv1771210747=41, srv114720162=4, srv177813736=42, srv1450941121=23, srv1715493962=36, srv1299357302=14, srv997281223=99} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:19,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:19,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:19,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:19,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:19,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:19,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:19,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-16 03:55:19,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-16 03:55:19,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-16 03:55:19,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-16 03:55:19,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-16 03:55:19,772 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-16 03:55:19,894 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:19,894 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-11-16 03:55:19,895 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=2.56866953943652E-5, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.6666666666666666E-4); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-11-16 03:55:20,160 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 379 ms to try 5000 different iterations.  Found a solution that moves 27 regions; Going from a computed imbalance of 2.56866953943652E-5 to a new imbalance of 5.831551034345205E-6. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.003); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:20,161 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1199535853:83 , srv1258281299:83 , srv1651741792:83 , srv1670492614:83 , srv1857308619:83 , srv2146436661:83 , srv704190946:83 , srv786734974:83 , srv1017270171:84 , srv1097472324:84 , srv1111073185:84 , srv1131591443:84 , srv114720162:84 , srv1153864153:84 , srv1157589761:84 , srv1168629486:84 , srv1219154223:84 , srv1285205353:84 , srv1299357302:84 , srv1301316565:84 , srv1308966345:84 , srv1315615445:84 , srv1360348456:84 , srv1365382056:84 , srv1400469239:84 , srv1405373269:84 , srv1450941121:84 , srv1479872889:84 , srv1541577908:84 , srv1565885030:84 , srv158151449:84 , srv1606275020:84 , srv1618989724:84 , srv1631081464:84 , srv165395571:84 , srv1673303422:84 , srv1687534418:84 , srv1715493962:84 , srv1715633435:84 , srv1745317707:84 , srv1766129494:84 , srv1770443827:84 , srv1771210747:84 , srv177813736:84 , srv1798421398:84 , srv1846851722:84 , srv1852948133:84 , srv1909150514:84 , srv1922890433:84 , srv1928423964:84 , srv1956433424:84 , srv1982531417:84 , srv199509390:84 , srv2041481489:84 , srv2072755643:84 , srv2123505604:84 , srv242809047:84 , srv303820221:84 , srv318653403:84 , srv339066005:84 , srv360147107:84 , srv375101733:84 , srv396658469:84 , srv402516341:84 , srv410788949:84 , srv416980128:84 , srv422409462:84 , srv454436883:84 , srv47043225:84 , srv482026286:84 , srv486440841:84 , srv512497620:84 , srv568694476:84 , srv590852417:84 , srv60558672:84 , srv616723546:84 , srv628269580:84 , srv642042101:84 , srv653964680:84 , srv753011170:84 , srv756367854:84 , srv814572518:84 , srv857329039:84 , srv863227998:84 , srv877956434:84 , srv889769592:84 , srv900424713:84 , srv904401792:84 , srv926041777:84 , srv968580510:84 , srv994293830:84 , srv1275418408:85 , srv1391980587:85 , srv1783464740:85 , srv230856210:85 , srv414745456:85 , srv606699751:85 , srv685002252:85 , srv997281223:85 , srv1197998020:684 }
2023-11-16 03:55:20,164 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:20,165 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1770443827=40, srv1308966345=16, srv360147107=63, srv303820221=60, srv1618989724=29, srv616723546=80, srv2072755643=55, srv653964680=83, srv410788949=67, srv1017270171=0, srv889769592=93, srv422409462=70, srv1219154223=10, srv199509390=53, srv877956434=92, srv47043225=72, srv900424713=94, srv482026286=73, srv242809047=59, srv2146436661=57, srv60558672=78, srv606699751=79, srv1405373269=22, srv1745317707=38, srv1982531417=52, srv1857308619=47, srv1798421398=44, srv1285205353=13, srv685002252=84, srv486440841=74, srv1258281299=11, srv1651741792=31, srv904401792=95, srv512497620=75, srv416980128=69, srv786734974=88, srv2041481489=54, srv756367854=87, srv968580510=97, srv1391980587=20, srv1111073185=2, srv230856210=58, srv1365382056=19, srv1909150514=48, srv994293830=98, srv165395571=32, srv339066005=62, srv1631081464=30, srv1097472324=1, srv1670492614=33, srv1846851722=45, srv396658469=65, srv1565885030=26, srv1687534418=35, srv1400469239=21, srv628269580=81, srv704190946=85, srv1673303422=34, srv1131591443=3, srv1301316565=15, srv1766129494=39, srv375101733=64, srv318653403=61, srv1360348456=18, srv1479872889=24, srv2123505604=56, srv857329039=90, srv814572518=89, srv1715633435=37, srv158151449=27, srv1783464740=43, srv1168629486=7, srv1157589761=6, srv414745456=68, srv454436883=71, srv590852417=77, srv926041777=96, srv1275418408=12, srv1852948133=46, srv753011170=86, srv1606275020=28, srv1956433424=51, srv1199535853=9, srv1928423964=50, srv642042101=82, srv863227998=91, srv1315615445=17, srv1541577908=25, srv568694476=76, srv402516341=66, srv1153864153=5, srv1922890433=49, srv1197998020=8, srv1771210747=41, srv114720162=4, srv177813736=42, srv1450941121=23, srv1715493962=36, srv1299357302=14, srv997281223=99} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:20,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:20,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:20,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:20,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:20,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-11-16 03:55:20,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-11-16 03:55:20,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-11-16 03:55:20,179 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-11-16 03:55:20,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-11-16 03:55:20,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-11-16 03:55:20,288 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=5.642492993526674E-6 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0012535023884839248); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:20,289 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1199535853:83 , srv1258281299:83 , srv1651741792:83 , srv1670492614:83 , srv1857308619:83 , srv2146436661:83 , srv704190946:83 , srv786734974:83 , srv1017270171:84 , srv1097472324:84 , srv1111073185:84 , srv1131591443:84 , srv114720162:84 , srv1153864153:84 , srv1157589761:84 , srv1168629486:84 , srv1219154223:84 , srv1285205353:84 , srv1299357302:84 , srv1301316565:84 , srv1308966345:84 , srv1315615445:84 , srv1360348456:84 , srv1365382056:84 , srv1400469239:84 , srv1405373269:84 , srv1450941121:84 , srv1479872889:84 , srv1541577908:84 , srv1565885030:84 , srv158151449:84 , srv1606275020:84 , srv1618989724:84 , srv1631081464:84 , srv165395571:84 , srv1673303422:84 , srv1687534418:84 , srv1715493962:84 , srv1715633435:84 , srv1745317707:84 , srv1766129494:84 , srv1770443827:84 , srv1771210747:84 , srv177813736:84 , srv1798421398:84 , srv1846851722:84 , srv1852948133:84 , srv1909150514:84 , srv1922890433:84 , srv1928423964:84 , srv1956433424:84 , srv1982531417:84 , srv199509390:84 , srv2041481489:84 , srv2072755643:84 , srv2123505604:84 , srv242809047:84 , srv303820221:84 , srv318653403:84 , srv339066005:84 , srv360147107:84 , srv375101733:84 , srv396658469:84 , srv402516341:84 , srv410788949:84 , srv416980128:84 , srv422409462:84 , srv454436883:84 , srv47043225:84 , srv482026286:84 , srv486440841:84 , srv512497620:84 , srv568694476:84 , srv590852417:84 , srv60558672:84 , srv616723546:84 , srv628269580:84 , srv642042101:84 , srv653964680:84 , srv753011170:84 , srv756367854:84 , srv814572518:84 , srv857329039:84 , srv863227998:84 , srv877956434:84 , srv889769592:84 , srv900424713:84 , srv904401792:84 , srv926041777:84 , srv968580510:84 , srv994293830:84 , srv1275418408:85 , srv1391980587:85 , srv1783464740:85 , srv230856210:85 , srv414745456:85 , srv606699751:85 , srv685002252:85 , srv997281223:85 , srv1197998020:684 }
2023-11-16 03:55:20,314 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=193 (was 193), ProcessCount=170 (was 170), AvailableMemoryMB=4516 (was 5028)
2023-11-16 03:55:20,322 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=193, ProcessCount=170, AvailableMemoryMB=4516
2023-11-16 03:55:20,322 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-11-16 03:55:20,323 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-11-16 03:55:20,333 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1047409924:84 , srv1051418080:84 , srv1083090441:84 , srv1176017569:84 , srv1220740826:84 , srv1353970355:84 , srv14474513:84 , srv1528751546:84 , srv159310802:84 , srv1695813882:84 , srv173472359:84 , srv1748503013:84 , srv1771078438:84 , srv1902296495:84 , srv1948203987:84 , srv2053374762:84 , srv229235755:84 , srv259472468:84 , srv283566644:84 , srv288195119:84 , srv334290994:84 , srv338548181:84 , srv633335207:84 , srv636228693:84 , srv777840796:84 , srv824350533:84 , srv884771548:84 , srv898979281:84 , srv995670567:84 , srv961918090:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-11-16 03:55:20,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:55:20,335 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1047409924=0, srv2053374762=15, srv1353970355=5, srv283566644=18, srv995670567=29, srv1748503013=11, srv1948203987=14, srv884771548=26, srv288195119=19, srv636228693=23, srv338548181=21, srv824350533=25, srv633335207=22, srv1695813882=9, srv961918090=28, srv1083090441=2, srv259472468=17, srv1051418080=1, srv334290994=20, srv1528751546=7, srv159310802=8, srv777840796=24, srv898979281=27, srv1902296495=13, srv173472359=10, srv229235755=16, srv14474513=6, srv1176017569=3, srv1220740826=4, srv1771078438=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:55:20,342 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:55:20,343 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-16 03:55:20,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0608439785709463, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:55:20,358 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912384760654019, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.0608439785709463, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-11-16 03:57:20,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 6162818 different iterations.  Found a solution that moves 1850 regions; Going from a computed imbalance of 0.9912384760654019 to a new imbalance of 4.317992290299789E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6851851851851852, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:57:20,354 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1047409924:90 , srv1051418080:90 , srv1083090441:90 , srv1176017569:90 , srv1220740826:90 , srv1353970355:90 , srv14474513:90 , srv1528751546:90 , srv159310802:90 , srv1695813882:90 , srv173472359:90 , srv1748503013:90 , srv1771078438:90 , srv1902296495:90 , srv1948203987:90 , srv2053374762:90 , srv229235755:90 , srv259472468:90 , srv283566644:90 , srv288195119:90 , srv334290994:90 , srv338548181:90 , srv633335207:90 , srv636228693:90 , srv777840796:90 , srv824350533:90 , srv884771548:90 , srv898979281:90 , srv961918090:90 , srv995670567:90 }
2023-11-16 03:57:20,355 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-11-16 03:57:20,356 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1047409924=0, srv2053374762=15, srv1353970355=5, srv283566644=18, srv995670567=29, srv1748503013=11, srv1948203987=14, srv884771548=26, srv288195119=19, srv636228693=23, srv338548181=21, srv824350533=25, srv633335207=22, srv1695813882=9, srv961918090=28, srv1083090441=2, srv259472468=17, srv1051418080=1, srv334290994=20, srv1528751546=7, srv159310802=8, srv777840796=24, srv898979281=27, srv1902296495=13, srv173472359=10, srv229235755=16, srv14474513=6, srv1176017569=3, srv1220740826=4, srv1771078438=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-11-16 03:57:20,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-11-16 03:57:20,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-11-16 03:57:20,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-11-16 03:57:20,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-11-16 03:57:20,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-11-16 03:57:20,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-11-16 03:57:20,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-11-16 03:57:20,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-11-16 03:57:20,378 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-11-16 03:57:20,378 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1047409924:90 , srv1051418080:90 , srv1083090441:90 , srv1176017569:90 , srv1220740826:90 , srv1353970355:90 , srv14474513:90 , srv1528751546:90 , srv159310802:90 , srv1695813882:90 , srv173472359:90 , srv1748503013:90 , srv1771078438:90 , srv1902296495:90 , srv1948203987:90 , srv2053374762:90 , srv229235755:90 , srv259472468:90 , srv283566644:90 , srv288195119:90 , srv334290994:90 , srv338548181:90 , srv633335207:90 , srv636228693:90 , srv777840796:90 , srv824350533:90 , srv884771548:90 , srv898979281:90 , srv961918090:90 , srv995670567:90 }
2023-11-16 03:57:20,391 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=290 (was 290), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 193), ProcessCount=172 (was 170) - ProcessCount LEAK? -, AvailableMemoryMB=4739 (was 4516) - AvailableMemoryMB LEAK? -