2023-02-18 03:56:53,114 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-02-18 03:56:53,386 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-02-18 03:56:53,500 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-02-18 03:56:53,551 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-18 03:56:53,557 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-18 03:56:53,567 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=164, AvailableMemoryMB=4159
2023-02-18 03:56:53,569 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-18 03:56:53,569 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-02-18 03:56:53,678 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1001654435:84 , srv1010263460:84 , srv1066538813:84 , srv1066628910:84 , srv1079077019:84 , srv1082704361:84 , srv1084384302:84 , srv1090362005:84 , srv1120583534:84 , srv1138707071:84 , srv1140366166:84 , srv115205832:84 , srv1224013765:84 , srv1255615919:84 , srv1284660482:84 , srv1290376694:84 , srv1302079351:84 , srv1324422570:84 , srv1329254904:84 , srv1359204319:84 , srv1390526117:84 , srv1392339524:84 , srv1395052813:84 , srv1399860600:84 , srv1475184343:84 , srv147825443:84 , srv1481907047:84 , srv1494348882:84 , srv1536430784:84 , srv1563841420:84 , srv157185974:84 , srv1608975058:84 , srv1610264324:84 , srv1616893746:84 , srv1620741932:84 , srv1642239206:84 , srv1668130524:84 , srv1744312913:84 , srv1746465618:84 , srv1796733636:84 , srv1806531408:84 , srv1826340727:84 , srv1849787751:84 , srv1856217869:84 , srv1879453682:84 , srv1884162083:84 , srv1885681202:84 , srv1903654384:84 , srv1918865217:84 , srv1963339204:84 , srv1979221647:84 , srv1983596244:84 , srv2000162505:84 , srv2008735741:84 , srv201528596:84 , srv2027604800:84 , srv2030755394:84 , srv203370255:84 , srv2046407877:84 , srv2068897567:84 , srv2075900327:84 , srv2108058864:84 , srv2120822571:84 , srv218797336:84 , srv223433428:84 , srv26047285:84 , srv266193506:84 , srv27061856:84 , srv27795495:84 , srv282189901:84 , srv29556601:84 , srv317317777:84 , srv324486783:84 , srv336269644:84 , srv338907630:84 , srv397521935:84 , srv423662646:84 , srv454665819:84 , srv459167804:84 , srv464858422:84 , srv482736825:84 , srv570267630:84 , srv608036316:84 , srv6454681:84 , srv661639342:84 , srv729757816:84 , srv734876489:84 , srv737685981:84 , srv74756093:84 , srv788514918:84 , srv821865959:84 , srv829888827:84 , srv850005904:84 , srv871362870:84 , srv872051953:84 , srv897393040:84 , srv900663356:84 , srv944967475:84 , srv971402463:84 , srv379668659:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-02-18 03:56:53,685 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:56:53,691 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1001654435=0, srv1918865217=48, srv2000162505=52, srv1796733636=39, srv1610264324=32, srv1979221647=50, srv336269644=73, srv324486783=72, srv608036316=83, srv829888827=92, srv1284660482=14, srv157185974=30, srv2008735741=53, srv1359204319=19, srv201528596=54, srv2046407877=58, srv788514918=90, srv1395052813=22, srv464858422=80, srv944967475=98, srv1620741932=34, srv1642239206=35, srv1494348882=27, srv1290376694=15, srv266193506=66, srv26047285=65, srv1120583534=8, srv737685981=88, srv821865959=91, srv218797336=63, srv6454681=84, srv423662646=77, srv223433428=64, srv1746465618=38, srv2030755394=56, srv2068897567=59, srv661639342=85, srv1963339204=49, srv397521935=76, srv317317777=71, srv1806531408=40, srv1536430784=28, srv2075900327=60, srv1392339524=21, srv570267630=82, srv1849787751=42, srv203370255=57, srv1090362005=7, srv1066538813=2, srv1616893746=33, srv1983596244=51, srv147825443=25, srv1084384302=6, srv338907630=74, srv1563841420=29, srv1324422570=17, srv1744312913=37, srv1138707071=9, srv1879453682=44, srv2108058864=61, srv482736825=81, srv729757816=86, srv379668659=75, srv1884162083=45, srv1475184343=24, srv1082704361=5, srv900663356=97, srv454665819=78, srv1010263460=1, srv74756093=89, srv872051953=95, srv282189901=69, srv850005904=93, srv1390526117=20, srv1329254904=18, srv971402463=99, srv1856217869=43, srv115205832=11, srv1302079351=16, srv1608975058=31, srv2027604800=55, srv27795495=68, srv1826340727=41, srv1885681202=46, srv1066628910=3, srv1399860600=23, srv1481907047=26, srv2120822571=62, srv27061856=67, srv897393040=96, srv1903654384=47, srv29556601=70, srv1079077019=4, srv1255615919=13, srv1668130524=36, srv1140366166=10, srv459167804=79, srv734876489=87, srv871362870=94, srv1224013765=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:56:53,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:56:53,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:56:53,758 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:56:53,759 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:56:53,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:56:53,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:56:53,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:56:53,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:56:53,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:56:53,760 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:56:53,761 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-18 03:56:53,762 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-18 03:56:53,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-18 03:56:53,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-18 03:56:53,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-18 03:56:53,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-18 03:56:53,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-18 03:56:53,763 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-18 03:56:53,764 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-18 03:56:53,765 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-18 03:56:53,766 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-18 03:56:53,767 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-18 03:56:53,768 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-18 03:56:53,769 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-18 03:56:53,770 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-18 03:56:53,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-18 03:56:53,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-18 03:56:53,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-18 03:56:53,771 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-18 03:56:53,907 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:53,907 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-18 03:56:53,907 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-18 03:56:54,132 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 333 ms to try 5000 different iterations.  Found a solution that moves 4648 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.18451051969296703. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04128500849792995, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055011477223054335, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5164444444444445, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04128500849792995, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17991666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.245, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:54,148 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1481907047:82 , srv1494348882:82 , srv1885681202:82 , srv1079077019:83 , srv2027604800:83 , srv203370255:83 , srv26047285:83 , srv317317777:83 , srv464858422:83 , srv74756093:83 , srv829888827:83 , srv944967475:83 , srv1138707071:84 , srv1399860600:84 , srv1744312913:84 , srv1879453682:84 , srv1979221647:84 , srv2030755394:84 , srv223433428:84 , srv397521935:84 , srv454665819:84 , srv897393040:84 , srv1084384302:85 , srv1140366166:85 , srv1475184343:85 , srv157185974:85 , srv1608975058:85 , srv1620741932:85 , srv1642239206:85 , srv1806531408:85 , srv1826340727:85 , srv1849787751:85 , srv1903654384:85 , srv2000162505:85 , srv2120822571:85 , srv27795495:85 , srv282189901:85 , srv29556601:85 , srv324486783:85 , srv482736825:85 , srv570267630:85 , srv788514918:85 , srv1001654435:86 , srv1066628910:86 , srv1082704361:86 , srv1224013765:86 , srv147825443:86 , srv1746465618:86 , srv1983596244:86 , srv336269644:86 , srv338907630:86 , srv459167804:86 , srv729757816:86 , srv871362870:86 , srv1010263460:87 , srv1066538813:87 , srv1090362005:87 , srv1120583534:87 , srv1255615919:87 , srv1302079351:87 , srv1359204319:87 , srv1390526117:87 , srv1392339524:87 , srv1395052813:87 , srv1610264324:87 , srv1668130524:87 , srv1856217869:87 , srv1884162083:87 , srv2008735741:87 , srv6454681:87 , srv661639342:87 , srv737685981:87 , srv850005904:87 , srv971402463:87 , srv1536430784:88 , srv1563841420:88 , srv1796733636:88 , srv1963339204:88 , srv2068897567:88 , srv2075900327:88 , srv218797336:88 , srv266193506:88 , srv608036316:88 , srv821865959:88 , srv1284660482:89 , srv1290376694:89 , srv1329254904:89 , srv1616893746:89 , srv1918865217:89 , srv201528596:89 , srv423662646:89 , srv872051953:89 , srv900663356:90 , srv1324422570:91 , srv2046407877:91 , srv2108058864:91 , srv27061856:91 , srv734876489:95 , srv115205832:97 , srv379668659:457 }
2023-02-18 03:56:54,154 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:56:54,155 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1001654435=0, srv1918865217=48, srv2000162505=52, srv1796733636=39, srv1610264324=32, srv1979221647=50, srv336269644=73, srv324486783=72, srv608036316=83, srv829888827=92, srv1284660482=14, srv157185974=30, srv2008735741=53, srv1359204319=19, srv201528596=54, srv2046407877=58, srv788514918=90, srv1395052813=22, srv464858422=80, srv944967475=98, srv1620741932=34, srv1642239206=35, srv1494348882=27, srv1290376694=15, srv266193506=66, srv26047285=65, srv1120583534=8, srv737685981=88, srv821865959=91, srv218797336=63, srv6454681=84, srv423662646=77, srv223433428=64, srv1746465618=38, srv2030755394=56, srv2068897567=59, srv661639342=85, srv1963339204=49, srv397521935=76, srv317317777=71, srv1806531408=40, srv1536430784=28, srv2075900327=60, srv1392339524=21, srv570267630=82, srv1849787751=42, srv203370255=57, srv1090362005=7, srv1066538813=2, srv1616893746=33, srv1983596244=51, srv147825443=25, srv1084384302=6, srv338907630=74, srv1563841420=29, srv1324422570=17, srv1744312913=37, srv1138707071=9, srv1879453682=44, srv2108058864=61, srv482736825=81, srv729757816=86, srv379668659=75, srv1884162083=45, srv1475184343=24, srv1082704361=5, srv900663356=97, srv454665819=78, srv1010263460=1, srv74756093=89, srv872051953=95, srv282189901=69, srv850005904=93, srv1390526117=20, srv1329254904=18, srv971402463=99, srv1856217869=43, srv115205832=11, srv1302079351=16, srv1608975058=31, srv2027604800=55, srv27795495=68, srv1826340727=41, srv1885681202=46, srv1066628910=3, srv1399860600=23, srv1481907047=26, srv2120822571=62, srv27061856=67, srv897393040=96, srv1903654384=47, srv29556601=70, srv1079077019=4, srv1255615919=13, srv1668130524=36, srv1140366166=10, srv459167804=79, srv734876489=87, srv871362870=94, srv1224013765=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:56:54,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:56:54,172 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-18 03:56:54,173 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-18 03:56:54,174 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-18 03:56:54,175 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-18 03:56:54,176 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-18 03:56:54,177 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-18 03:56:54,178 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-18 03:56:54,279 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04128500849792995, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055011477223054335, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04128500849792995, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17991666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.245, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:54,279 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-18 03:56:54,280 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.1844779737013476, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.04128500849792995, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.055011477223054335, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.04128500849792995, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.17991666666666667, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.245, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-18 03:56:54,467 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 275 ms to try 5000 different iterations.  Found a solution that moves 2635 regions; Going from a computed imbalance of 0.1844779737013476 to a new imbalance of 0.01661018819557537. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023417889653104165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03753038424880666, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2927777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023417889653104165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:54,476 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv115205832:75 , srv734876489:78 , srv1324422570:80 , srv2046407877:80 , srv872051953:80 , srv1284660482:81 , srv2108058864:81 , srv27061856:81 , srv1616893746:82 , srv2075900327:82 , srv1120583534:83 , srv1255615919:83 , srv1329254904:83 , srv1390526117:83 , srv1563841420:83 , srv1856217869:83 , srv1884162083:83 , srv1963339204:83 , srv2068897567:83 , srv900663356:83 , srv1290376694:84 , srv1359204319:84 , srv1392339524:84 , srv1536430784:84 , srv1796733636:84 , srv2008735741:84 , srv201528596:84 , srv423662646:84 , srv6454681:84 , srv1066538813:85 , srv1066628910:85 , srv1082704361:85 , srv1395052813:85 , srv1399860600:85 , srv1608975058:85 , srv1668130524:85 , srv1806531408:85 , srv1826340727:85 , srv1849787751:85 , srv1903654384:85 , srv1918865217:85 , srv1983596244:85 , srv218797336:85 , srv282189901:85 , srv821865959:85 , srv1001654435:86 , srv1010263460:86 , srv1084384302:86 , srv1090362005:86 , srv1138707071:86 , srv1140366166:86 , srv1224013765:86 , srv1610264324:86 , srv459167804:86 , srv608036316:86 , srv737685981:86 , srv74756093:86 , srv829888827:86 , srv971402463:86 , srv1302079351:87 , srv1475184343:87 , srv147825443:87 , srv157185974:87 , srv1885681202:87 , srv2000162505:87 , srv2027604800:87 , srv2030755394:87 , srv203370255:87 , srv2120822571:87 , srv223433428:87 , srv266193506:87 , srv29556601:87 , srv317317777:87 , srv336269644:87 , srv338907630:87 , srv397521935:87 , srv454665819:87 , srv570267630:87 , srv850005904:87 , srv897393040:87 , srv1642239206:88 , srv1744312913:88 , srv27795495:88 , srv324486783:88 , srv482736825:88 , srv1481907047:89 , srv1746465618:89 , srv1879453682:89 , srv1979221647:89 , srv26047285:89 , srv729757816:89 , srv788514918:89 , srv1079077019:90 , srv1620741932:90 , srv661639342:90 , srv1494348882:91 , srv464858422:91 , srv871362870:91 , srv944967475:92 , srv379668659:525 }
2023-02-18 03:56:54,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:56:54,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1001654435=0, srv1918865217=48, srv2000162505=52, srv1796733636=39, srv1610264324=32, srv1979221647=50, srv336269644=73, srv324486783=72, srv608036316=83, srv829888827=92, srv1284660482=14, srv157185974=30, srv2008735741=53, srv1359204319=19, srv201528596=54, srv2046407877=58, srv788514918=90, srv1395052813=22, srv464858422=80, srv944967475=98, srv1620741932=34, srv1642239206=35, srv1494348882=27, srv1290376694=15, srv266193506=66, srv26047285=65, srv1120583534=8, srv737685981=88, srv821865959=91, srv218797336=63, srv6454681=84, srv423662646=77, srv223433428=64, srv1746465618=38, srv2030755394=56, srv2068897567=59, srv661639342=85, srv1963339204=49, srv397521935=76, srv317317777=71, srv1806531408=40, srv1536430784=28, srv2075900327=60, srv1392339524=21, srv570267630=82, srv1849787751=42, srv203370255=57, srv1090362005=7, srv1066538813=2, srv1616893746=33, srv1983596244=51, srv147825443=25, srv1084384302=6, srv338907630=74, srv1563841420=29, srv1324422570=17, srv1744312913=37, srv1138707071=9, srv1879453682=44, srv2108058864=61, srv482736825=81, srv729757816=86, srv379668659=75, srv1884162083=45, srv1475184343=24, srv1082704361=5, srv900663356=97, srv454665819=78, srv1010263460=1, srv74756093=89, srv872051953=95, srv282189901=69, srv850005904=93, srv1390526117=20, srv1329254904=18, srv971402463=99, srv1856217869=43, srv115205832=11, srv1302079351=16, srv1608975058=31, srv2027604800=55, srv27795495=68, srv1826340727=41, srv1885681202=46, srv1066628910=3, srv1399860600=23, srv1481907047=26, srv2120822571=62, srv27061856=67, srv897393040=96, srv1903654384=47, srv29556601=70, srv1079077019=4, srv1255615919=13, srv1668130524=36, srv1140366166=10, srv459167804=79, srv734876489=87, srv871362870=94, srv1224013765=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:56:54,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:56:54,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:56:54,493 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:56:54,494 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:56:54,495 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-18 03:56:54,496 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-18 03:56:54,497 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-18 03:56:54,498 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-18 03:56:54,499 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-18 03:56:54,500 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-18 03:56:54,501 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-18 03:56:54,593 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023417889653104165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03753038424880666, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023417889653104165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:54,593 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-18 03:56:54,594 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.016591737531221413, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.023417889653104165); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03753038424880666, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.023417889653104165); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.013666666666666667); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0445, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-18 03:56:54,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 296 ms to try 5000 different iterations.  Found a solution that moves 1006 regions; Going from a computed imbalance of 0.016591737531221413 to a new imbalance of 8.607507209407857E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005986641820279941); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006581953356624206); MoveCostFunction : (multiplier=7.0, imbalance=0.11177777777777778, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005986641820279941); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:54,808 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv661639342:80 , srv871362870:81 , srv944967475:81 , srv1746465618:82 , srv266193506:82 , srv1302079351:83 , srv1620741932:83 , srv608036316:83 , srv729757816:83 , srv734876489:83 , srv850005904:83 , srv1010263460:84 , srv1290376694:84 , srv147825443:84 , srv1642239206:84 , srv1879453682:84 , srv1918865217:84 , srv218797336:84 , srv336269644:84 , srv338907630:84 , srv464858422:84 , srv482736825:84 , srv788514918:84 , srv971402463:84 , srv1079077019:85 , srv1224013765:85 , srv1494348882:85 , srv1796733636:85 , srv1979221647:85 , srv201528596:85 , srv2108058864:85 , srv2120822571:85 , srv26047285:85 , srv27061856:85 , srv27795495:85 , srv324486783:85 , srv423662646:85 , srv459167804:85 , srv737685981:85 , srv821865959:85 , srv900663356:85 , srv1001654435:86 , srv1066538813:86 , srv1066628910:86 , srv1082704361:86 , srv1090362005:86 , srv115205832:86 , srv1329254904:86 , srv1359204319:86 , srv1392339524:86 , srv1395052813:86 , srv1475184343:86 , srv1536430784:86 , srv1563841420:86 , srv157185974:86 , srv1610264324:86 , srv1616893746:86 , srv1668130524:86 , srv1963339204:86 , srv1983596244:86 , srv2000162505:86 , srv2046407877:86 , srv2068897567:86 , srv223433428:86 , srv29556601:86 , srv454665819:86 , srv570267630:86 , srv6454681:86 , srv897393040:86 , srv1084384302:87 , srv1120583534:87 , srv1138707071:87 , srv1140366166:87 , srv1284660482:87 , srv1324422570:87 , srv1390526117:87 , srv1481907047:87 , srv1608975058:87 , srv1744312913:87 , srv1806531408:87 , srv1826340727:87 , srv1856217869:87 , srv1884162083:87 , srv2008735741:87 , srv2030755394:87 , srv203370255:87 , srv2075900327:87 , srv317317777:87 , srv1255615919:88 , srv1399860600:88 , srv1849787751:88 , srv1885681202:88 , srv1903654384:88 , srv2027604800:88 , srv282189901:88 , srv397521935:88 , srv829888827:88 , srv872051953:88 , srv74756093:89 , srv379668659:529 }
2023-02-18 03:56:54,813 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:56:54,814 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1001654435=0, srv1918865217=48, srv2000162505=52, srv1796733636=39, srv1610264324=32, srv1979221647=50, srv336269644=73, srv324486783=72, srv608036316=83, srv829888827=92, srv1284660482=14, srv157185974=30, srv2008735741=53, srv1359204319=19, srv201528596=54, srv2046407877=58, srv788514918=90, srv1395052813=22, srv464858422=80, srv944967475=98, srv1620741932=34, srv1642239206=35, srv1494348882=27, srv1290376694=15, srv266193506=66, srv26047285=65, srv1120583534=8, srv737685981=88, srv821865959=91, srv218797336=63, srv6454681=84, srv423662646=77, srv223433428=64, srv1746465618=38, srv2030755394=56, srv2068897567=59, srv661639342=85, srv1963339204=49, srv397521935=76, srv317317777=71, srv1806531408=40, srv1536430784=28, srv2075900327=60, srv1392339524=21, srv570267630=82, srv1849787751=42, srv203370255=57, srv1090362005=7, srv1066538813=2, srv1616893746=33, srv1983596244=51, srv147825443=25, srv1084384302=6, srv338907630=74, srv1563841420=29, srv1324422570=17, srv1744312913=37, srv1138707071=9, srv1879453682=44, srv2108058864=61, srv482736825=81, srv729757816=86, srv379668659=75, srv1884162083=45, srv1475184343=24, srv1082704361=5, srv900663356=97, srv454665819=78, srv1010263460=1, srv74756093=89, srv872051953=95, srv282189901=69, srv850005904=93, srv1390526117=20, srv1329254904=18, srv971402463=99, srv1856217869=43, srv115205832=11, srv1302079351=16, srv1608975058=31, srv2027604800=55, srv27795495=68, srv1826340727=41, srv1885681202=46, srv1066628910=3, srv1399860600=23, srv1481907047=26, srv2120822571=62, srv27061856=67, srv897393040=96, srv1903654384=47, srv29556601=70, srv1079077019=4, srv1255615919=13, srv1668130524=36, srv1140366166=10, srv459167804=79, srv734876489=87, srv871362870=94, srv1224013765=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:56:54,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:56:54,825 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:56:54,826 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:56:54,827 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-18 03:56:54,828 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-18 03:56:54,829 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-18 03:56:54,830 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-18 03:56:54,831 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-18 03:56:54,832 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-18 03:56:54,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005986641820279941); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006581953356624206); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005986641820279941); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:54,937 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-02-18 03:56:54,937 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=8.537065583828803E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005986641820279941); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.006581953356624206); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005986641820279941); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=5.0E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.003833333333333333); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-02-18 03:56:55,166 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 325 ms to try 5000 different iterations.  Found a solution that moves 227 regions; Going from a computed imbalance of 8.537065583828803E-4 to a new imbalance of 2.0539290259610263E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.025222222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:55,167 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1090362005:83 , srv1610264324:83 , srv1744312913:83 , srv397521935:83 , srv821865959:83 , srv1001654435:84 , srv1066538813:84 , srv1079077019:84 , srv1084384302:84 , srv1140366166:84 , srv115205832:84 , srv1255615919:84 , srv1324422570:84 , srv1329254904:84 , srv1395052813:84 , srv1475184343:84 , srv1481907047:84 , srv1536430784:84 , srv157185974:84 , srv1620741932:84 , srv1668130524:84 , srv1826340727:84 , srv1856217869:84 , srv1903654384:84 , srv1918865217:84 , srv1979221647:84 , srv2000162505:84 , srv2008735741:84 , srv2027604800:84 , srv2030755394:84 , srv2075900327:84 , srv27795495:84 , srv282189901:84 , srv317317777:84 , srv324486783:84 , srv423662646:84 , srv464858422:84 , srv570267630:84 , srv608036316:84 , srv6454681:84 , srv729757816:84 , srv74756093:84 , srv788514918:84 , srv900663356:84 , srv971402463:84 , srv1010263460:85 , srv1066628910:85 , srv1082704361:85 , srv1120583534:85 , srv1138707071:85 , srv1224013765:85 , srv1284660482:85 , srv1290376694:85 , srv1302079351:85 , srv1359204319:85 , srv1390526117:85 , srv1392339524:85 , srv1399860600:85 , srv1494348882:85 , srv1563841420:85 , srv1608975058:85 , srv1616893746:85 , srv1642239206:85 , srv1746465618:85 , srv1796733636:85 , srv1806531408:85 , srv1849787751:85 , srv1879453682:85 , srv1884162083:85 , srv1885681202:85 , srv1963339204:85 , srv1983596244:85 , srv201528596:85 , srv203370255:85 , srv2046407877:85 , srv2068897567:85 , srv2108058864:85 , srv2120822571:85 , srv218797336:85 , srv223433428:85 , srv26047285:85 , srv266193506:85 , srv27061856:85 , srv29556601:85 , srv336269644:85 , srv454665819:85 , srv459167804:85 , srv482736825:85 , srv661639342:85 , srv737685981:85 , srv829888827:85 , srv850005904:85 , srv871362870:85 , srv872051953:85 , srv897393040:85 , srv147825443:86 , srv338907630:86 , srv734876489:86 , srv944967475:86 , srv379668659:631 }
2023-02-18 03:56:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:56:55,171 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1001654435=0, srv1918865217=48, srv2000162505=52, srv1796733636=39, srv1610264324=32, srv1979221647=50, srv336269644=73, srv324486783=72, srv608036316=83, srv829888827=92, srv1284660482=14, srv157185974=30, srv2008735741=53, srv1359204319=19, srv201528596=54, srv2046407877=58, srv788514918=90, srv1395052813=22, srv464858422=80, srv944967475=98, srv1620741932=34, srv1642239206=35, srv1494348882=27, srv1290376694=15, srv266193506=66, srv26047285=65, srv1120583534=8, srv737685981=88, srv821865959=91, srv218797336=63, srv6454681=84, srv423662646=77, srv223433428=64, srv1746465618=38, srv2030755394=56, srv2068897567=59, srv661639342=85, srv1963339204=49, srv397521935=76, srv317317777=71, srv1806531408=40, srv1536430784=28, srv2075900327=60, srv1392339524=21, srv570267630=82, srv1849787751=42, srv203370255=57, srv1090362005=7, srv1066538813=2, srv1616893746=33, srv1983596244=51, srv147825443=25, srv1084384302=6, srv338907630=74, srv1563841420=29, srv1324422570=17, srv1744312913=37, srv1138707071=9, srv1879453682=44, srv2108058864=61, srv482736825=81, srv729757816=86, srv379668659=75, srv1884162083=45, srv1475184343=24, srv1082704361=5, srv900663356=97, srv454665819=78, srv1010263460=1, srv74756093=89, srv872051953=95, srv282189901=69, srv850005904=93, srv1390526117=20, srv1329254904=18, srv971402463=99, srv1856217869=43, srv115205832=11, srv1302079351=16, srv1608975058=31, srv2027604800=55, srv27795495=68, srv1826340727=41, srv1885681202=46, srv1066628910=3, srv1399860600=23, srv1481907047=26, srv2120822571=62, srv27061856=67, srv897393040=96, srv1903654384=47, srv29556601=70, srv1079077019=4, srv1255615919=13, srv1668130524=36, srv1140366166=10, srv459167804=79, srv734876489=87, srv871362870=94, srv1224013765=12} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:56:55,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:56:55,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:56:55,180 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:56:55,181 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-02-18 03:56:55,182 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-02-18 03:56:55,183 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-02-18 03:56:55,184 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-02-18 03:56:55,185 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-02-18 03:56:55,186 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-02-18 03:56:55,272 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.894980228680261E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.4668347344853864E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0020651563939657444); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.4668347344853864E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:55,272 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1090362005:83 , srv1610264324:83 , srv1744312913:83 , srv397521935:83 , srv821865959:83 , srv1001654435:84 , srv1066538813:84 , srv1079077019:84 , srv1084384302:84 , srv1140366166:84 , srv115205832:84 , srv1255615919:84 , srv1324422570:84 , srv1329254904:84 , srv1395052813:84 , srv1475184343:84 , srv1481907047:84 , srv1536430784:84 , srv157185974:84 , srv1620741932:84 , srv1668130524:84 , srv1826340727:84 , srv1856217869:84 , srv1903654384:84 , srv1918865217:84 , srv1979221647:84 , srv2000162505:84 , srv2008735741:84 , srv2027604800:84 , srv2030755394:84 , srv2075900327:84 , srv27795495:84 , srv282189901:84 , srv317317777:84 , srv324486783:84 , srv423662646:84 , srv464858422:84 , srv570267630:84 , srv608036316:84 , srv6454681:84 , srv729757816:84 , srv74756093:84 , srv788514918:84 , srv900663356:84 , srv971402463:84 , srv1010263460:85 , srv1066628910:85 , srv1082704361:85 , srv1120583534:85 , srv1138707071:85 , srv1224013765:85 , srv1284660482:85 , srv1290376694:85 , srv1302079351:85 , srv1359204319:85 , srv1390526117:85 , srv1392339524:85 , srv1399860600:85 , srv1494348882:85 , srv1563841420:85 , srv1608975058:85 , srv1616893746:85 , srv1642239206:85 , srv1746465618:85 , srv1796733636:85 , srv1806531408:85 , srv1849787751:85 , srv1879453682:85 , srv1884162083:85 , srv1885681202:85 , srv1963339204:85 , srv1983596244:85 , srv201528596:85 , srv203370255:85 , srv2046407877:85 , srv2068897567:85 , srv2108058864:85 , srv2120822571:85 , srv218797336:85 , srv223433428:85 , srv26047285:85 , srv266193506:85 , srv27061856:85 , srv29556601:85 , srv336269644:85 , srv454665819:85 , srv459167804:85 , srv482736825:85 , srv661639342:85 , srv737685981:85 , srv829888827:85 , srv850005904:85 , srv871362870:85 , srv872051953:85 , srv897393040:85 , srv147825443:86 , srv338907630:86 , srv734876489:86 , srv944967475:86 , srv379668659:631 }
2023-02-18 03:56:55,300 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=10 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=291 (was 291), ProcessCount=164 (was 164), AvailableMemoryMB=3674 (was 4159)
2023-02-18 03:56:55,308 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=10, OpenFileDescriptor=260, MaxFileDescriptor=60000, SystemLoadAverage=291, ProcessCount=164, AvailableMemoryMB=3673
2023-02-18 03:56:55,308 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-02-18 03:56:55,308 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-02-18 03:56:55,318 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1083163758:84 , srv1141845630:84 , srv1148698515:84 , srv1214485542:84 , srv1512001603:84 , srv1604677324:84 , srv1754069649:84 , srv1788133778:84 , srv1834508720:84 , srv1856338726:84 , srv1860063425:84 , srv1990225493:84 , srv1993688297:84 , srv2051986799:84 , srv2131771989:84 , srv253582672:84 , srv388389810:84 , srv39562213:84 , srv423394438:84 , srv479438311:84 , srv527964986:84 , srv600307096:84 , srv732028710:84 , srv742945242:84 , srv752449066:84 , srv789627856:84 , srv799692214:84 , srv801087327:84 , srv990700304:84 , srv1621475000:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-02-18 03:56:55,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:56:55,320 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1788133778=8, srv990700304=29, srv1214485542=3, srv1990225493=12, srv388389810=17, srv253582672=16, srv1860063425=11, srv1993688297=13, srv1141845630=1, srv801087327=28, srv732028710=23, srv1604677324=5, srv799692214=27, srv2051986799=14, srv2131771989=15, srv1856338726=10, srv600307096=22, srv1754069649=7, srv789627856=26, srv752449066=25, srv423394438=19, srv1083163758=0, srv1834508720=9, srv1621475000=6, srv527964986=21, srv1512001603=4, srv742945242=24, srv479438311=20, srv1148698515=2, srv39562213=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:56:55,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:56:55,330 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:56:55,331 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:56:55,332 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-18 03:56:55,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1088557668933918, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:56:55,346 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912536044231293, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1088557668933918, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-02-18 03:58:55,335 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 9181809 different iterations.  Found a solution that moves 1840 regions; Going from a computed imbalance of 0.9912536044231293 to a new imbalance of 4.2946517914333035E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6814814814814815, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:58:55,340 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1083163758:90 , srv1141845630:90 , srv1148698515:90 , srv1214485542:90 , srv1512001603:90 , srv1604677324:90 , srv1621475000:90 , srv1754069649:90 , srv1788133778:90 , srv1834508720:90 , srv1856338726:90 , srv1860063425:90 , srv1990225493:90 , srv1993688297:90 , srv2051986799:90 , srv2131771989:90 , srv253582672:90 , srv388389810:90 , srv39562213:90 , srv423394438:90 , srv479438311:90 , srv527964986:90 , srv600307096:90 , srv732028710:90 , srv742945242:90 , srv752449066:90 , srv789627856:90 , srv799692214:90 , srv801087327:90 , srv990700304:90 }
2023-02-18 03:58:55,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-02-18 03:58:55,341 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1788133778=8, srv990700304=29, srv1214485542=3, srv1990225493=12, srv388389810=17, srv253582672=16, srv1860063425=11, srv1993688297=13, srv1141845630=1, srv801087327=28, srv732028710=23, srv1604677324=5, srv799692214=27, srv2051986799=14, srv2131771989=15, srv1856338726=10, srv600307096=22, srv1754069649=7, srv789627856=26, srv752449066=25, srv423394438=19, srv1083163758=0, srv1834508720=9, srv1621475000=6, srv527964986=21, srv1512001603=4, srv742945242=24, srv479438311=20, srv1148698515=2, srv39562213=18} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-02-18 03:58:55,344 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-02-18 03:58:55,345 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-02-18 03:58:55,355 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-02-18 03:58:55,355 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1083163758:90 , srv1141845630:90 , srv1148698515:90 , srv1214485542:90 , srv1512001603:90 , srv1604677324:90 , srv1621475000:90 , srv1754069649:90 , srv1788133778:90 , srv1834508720:90 , srv1856338726:90 , srv1860063425:90 , srv1990225493:90 , srv1993688297:90 , srv2051986799:90 , srv2131771989:90 , srv253582672:90 , srv388389810:90 , srv39562213:90 , srv423394438:90 , srv479438311:90 , srv527964986:90 , srv600307096:90 , srv732028710:90 , srv742945242:90 , srv752449066:90 , srv789627856:90 , srv799692214:90 , srv801087327:90 , srv990700304:90 }
2023-02-18 03:58:55,366 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=9 (was 10), OpenFileDescriptor=260 (was 260), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=303 (was 291) - SystemLoadAverage LEAK? -, ProcessCount=164 (was 164), AvailableMemoryMB=3812 (was 3673) - AvailableMemoryMB LEAK? -