2023-09-28 23:53:17,580 INFO  [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks timeout: 13 mins
2023-09-28 23:53:17,906 WARN  [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties
2023-09-28 23:53:18,022 INFO  [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl
2023-09-28 23:53:18,090 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-28 23:53:18,098 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-28 23:53:18,113 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=340, ProcessCount=173, AvailableMemoryMB=6554
2023-09-28 23:53:18,116 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-28 23:53:18,116 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=10000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc.
2023-09-28 23:53:18,251 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1019786092:84 , srv1034919098:84 , srv1042507277:84 , srv1052348479:84 , srv1059702692:84 , srv1109134663:84 , srv1111061847:84 , srv1118297118:84 , srv1127047112:84 , srv1139330443:84 , srv1189991976:84 , srv1196513703:84 , srv1220897953:84 , srv1234121085:84 , srv1269622124:84 , srv1306169651:84 , srv1311068106:84 , srv1325246387:84 , srv1334675084:84 , srv13349597:84 , srv1343818347:84 , srv1367302916:84 , srv1369938939:84 , srv1445122991:84 , srv150452741:84 , srv1513275022:84 , srv1542664134:84 , srv1545972677:84 , srv1578406846:84 , srv1665300419:84 , srv1667970193:84 , srv1693927348:84 , srv1716540845:84 , srv1717138708:84 , srv1766371739:84 , srv1805791762:84 , srv1885627163:84 , srv1922582811:84 , srv1925602737:84 , srv1944813605:84 , srv1951281289:84 , srv1959626483:84 , srv1974629959:84 , srv1991980814:84 , srv2044260841:84 , srv205788975:84 , srv205864601:84 , srv2061276039:84 , srv2067289538:84 , srv2075192617:84 , srv2077091875:84 , srv2081287264:84 , srv2084933964:84 , srv2089368072:84 , srv2112562618:84 , srv212631497:84 , srv2129429096:84 , srv228596705:84 , srv246794919:84 , srv25200564:84 , srv26151557:84 , srv262687856:84 , srv275896957:84 , srv316617436:84 , srv325424806:84 , srv354540686:84 , srv355682952:84 , srv372070482:84 , srv384520925:84 , srv41563228:84 , srv429079866:84 , srv471078577:84 , srv472766287:84 , srv476217809:84 , srv47951916:84 , srv482084472:84 , srv48373633:84 , srv484509358:84 , srv575329483:84 , srv605047709:84 , srv622068930:84 , srv622294977:84 , srv649499358:84 , srv708518145:84 , srv718053690:84 , srv723689657:84 , srv739438090:84 , srv744126112:84 , srv768924027:84 , srv799117480:84 , srv820935808:84 , srv823371050:84 , srv829272143:84 , srv851560390:84 , srv898974544:84 , srv913587994:84 , srv965888102:84 , srv986285548:84 , srv990370035:84 , srv2075619821:684 } [srvr=100 rgns=9000 avg=90.0 max=90 min=90]
2023-09-28 23:53:18,264 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:53:18,273 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv205864601=46, srv316617436=64, srv26151557=61, srv986285548=98, srv2112562618=55, srv1311068106=16, srv1118297118=7, srv1109134663=5, srv1059702692=4, srv1693927348=31, srv1991980814=43, srv1234121085=13, srv25200564=60, srv2075192617=49, srv1325246387=17, srv355682952=67, srv1369938939=22, srv2075619821=50, srv622294977=82, srv575329483=79, srv1052348479=3, srv246794919=59, srv275896957=63, srv820935808=91, srv384520925=69, srv1367302916=21, srv1665300419=29, srv1974629959=42, srv1542664134=26, srv1189991976=10, srv913587994=96, srv2067289538=48, srv1545972677=27, srv1885627163=36, srv2089368072=54, srv1042507277=2, srv372070482=68, srv13349597=19, srv482084472=76, srv476217809=74, srv1220897953=12, srv1717138708=33, srv1716540845=32, srv622068930=81, srv429079866=71, srv1578406846=28, srv1667970193=30, srv1445122991=23, srv1306169651=15, srv1034919098=1, srv262687856=62, srv823371050=92, srv1959626483=41, srv744126112=88, srv472766287=73, srv471078577=72, srv325424806=65, srv1127047112=8, srv723689657=86, srv898974544=95, srv1805791762=35, srv1944813605=39, srv2084933964=53, srv768924027=89, srv990370035=99, srv1922582811=37, srv1019786092=0, srv851560390=94, srv205788975=45, srv2077091875=51, srv2081287264=52, srv1925602737=38, srv1139330443=9, srv212631497=56, srv1343818347=20, srv2061276039=47, srv2129429096=57, srv1111061847=6, srv829272143=93, srv649499358=83, srv1196513703=11, srv2044260841=44, srv718053690=85, srv1766371739=34, srv1269622124=14, srv47951916=75, srv150452741=24, srv1513275022=25, srv48373633=77, srv605047709=80, srv739438090=87, srv1334675084=18, srv228596705=58, srv484509358=78, srv41563228=70, srv799117480=90, srv1951281289=40, srv965888102=97, srv354540686=66, srv708518145=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:53:18,358 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:53:18,359 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:53:18,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:53:18,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:53:18,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:53:18,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:53:18,360 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:53:18,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:53:18,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:53:18,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:53:18,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:53:18,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:53:18,361 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:53:18,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:53:18,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:53:18,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:53:18,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 23:53:18,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 23:53:18,362 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 23:53:18,363 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 23:53:18,364 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 23:53:18,365 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 23:53:18,366 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 23:53:18,367 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 23:53:18,368 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 23:53:18,369 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 23:53:18,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 23:53:18,370 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 23:53:18,572 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:18,573 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 23:53:18,573 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9909252140407105, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.06666666666666667, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 23:53:18,950 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 549 ms to try 5000 different iterations.  Found a solution that moves 4681 regions; Going from a computed imbalance of 0.9909252140407105 to a new imbalance of 0.17959745048530049. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03794115329830114, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05708009219336976, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.5201111111111111, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03794115329830114, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:18,967 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1665300419:80 , srv41563228:81 , srv1343818347:82 , srv1445122991:82 , srv1717138708:82 , srv649499358:82 , srv1306169651:83 , srv1311068106:83 , srv1693927348:83 , srv205788975:83 , srv2081287264:83 , srv472766287:83 , srv1118297118:84 , srv1367302916:84 , srv150452741:84 , srv1944813605:84 , srv2061276039:84 , srv429079866:84 , srv47951916:84 , srv1019786092:85 , srv1034919098:85 , srv1052348479:85 , srv1059702692:85 , srv1109134663:85 , srv1189991976:85 , srv1269622124:85 , srv1578406846:85 , srv1805791762:85 , srv228596705:85 , srv25200564:85 , srv26151557:85 , srv262687856:85 , srv354540686:85 , srv739438090:85 , srv823371050:85 , srv1042507277:86 , srv1196513703:86 , srv1369938939:86 , srv1542664134:86 , srv1667970193:86 , srv1716540845:86 , srv1959626483:86 , srv1991980814:86 , srv2044260841:86 , srv2067289538:86 , srv2089368072:86 , srv275896957:86 , srv476217809:86 , srv605047709:86 , srv718053690:86 , srv768924027:86 , srv829272143:86 , srv1127047112:87 , srv1139330443:87 , srv1234121085:87 , srv13349597:87 , srv1513275022:87 , srv1885627163:87 , srv1974629959:87 , srv212631497:87 , srv2129429096:87 , srv246794919:87 , srv372070482:87 , srv723689657:87 , srv744126112:87 , srv1325246387:88 , srv1545972677:88 , srv1922582811:88 , srv1951281289:88 , srv205864601:88 , srv325424806:88 , srv48373633:88 , srv622068930:88 , srv622294977:88 , srv820935808:88 , srv851560390:88 , srv990370035:88 , srv1925602737:89 , srv384520925:89 , srv482084472:89 , srv913587994:89 , srv965888102:89 , srv986285548:89 , srv1220897953:90 , srv1766371739:90 , srv2075192617:90 , srv316617436:90 , srv355682952:90 , srv484509358:90 , srv708518145:90 , srv898974544:90 , srv1334675084:91 , srv2084933964:91 , srv2112562618:91 , srv471078577:91 , srv2077091875:92 , srv575329483:92 , srv799117480:92 , srv1111061847:95 , srv2075619821:427 }
2023-09-28 23:53:18,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:53:18,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv205864601=46, srv316617436=64, srv26151557=61, srv986285548=98, srv2112562618=55, srv1311068106=16, srv1118297118=7, srv1109134663=5, srv1059702692=4, srv1693927348=31, srv1991980814=43, srv1234121085=13, srv25200564=60, srv2075192617=49, srv1325246387=17, srv355682952=67, srv1369938939=22, srv2075619821=50, srv622294977=82, srv575329483=79, srv1052348479=3, srv246794919=59, srv275896957=63, srv820935808=91, srv384520925=69, srv1367302916=21, srv1665300419=29, srv1974629959=42, srv1542664134=26, srv1189991976=10, srv913587994=96, srv2067289538=48, srv1545972677=27, srv1885627163=36, srv2089368072=54, srv1042507277=2, srv372070482=68, srv13349597=19, srv482084472=76, srv476217809=74, srv1220897953=12, srv1717138708=33, srv1716540845=32, srv622068930=81, srv429079866=71, srv1578406846=28, srv1667970193=30, srv1445122991=23, srv1306169651=15, srv1034919098=1, srv262687856=62, srv823371050=92, srv1959626483=41, srv744126112=88, srv472766287=73, srv471078577=72, srv325424806=65, srv1127047112=8, srv723689657=86, srv898974544=95, srv1805791762=35, srv1944813605=39, srv2084933964=53, srv768924027=89, srv990370035=99, srv1922582811=37, srv1019786092=0, srv851560390=94, srv205788975=45, srv2077091875=51, srv2081287264=52, srv1925602737=38, srv1139330443=9, srv212631497=56, srv1343818347=20, srv2061276039=47, srv2129429096=57, srv1111061847=6, srv829272143=93, srv649499358=83, srv1196513703=11, srv2044260841=44, srv718053690=85, srv1766371739=34, srv1269622124=14, srv47951916=75, srv150452741=24, srv1513275022=25, srv48373633=77, srv605047709=80, srv739438090=87, srv1334675084=18, srv228596705=58, srv484509358=78, srv41563228=70, srv799117480=90, srv1951281289=40, srv965888102=97, srv354540686=66, srv708518145=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:53:18,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:53:18,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:53:18,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:53:18,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:53:18,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:53:18,988 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:53:18,989 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 23:53:18,990 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 23:53:18,991 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 23:53:18,992 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 23:53:18,993 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 23:53:18,994 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 23:53:18,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 23:53:19,151 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03794115329830114, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05708009219336976, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03794115329830114, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:19,151 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 23:53:19,152 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.17956467342274227, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03794115329830114, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05708009219336976, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.03794115329830114, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.1745, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.24466666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 23:53:19,426 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 411 ms to try 5000 different iterations.  Found a solution that moves 2675 regions; Going from a computed imbalance of 0.17956467342274227 to a new imbalance of 0.014758548499791756. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02015960755521166); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03460535889421652, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.2972222222222222, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02015960755521166); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:19,437 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1111061847:77 , srv471078577:79 , srv2077091875:80 , srv2084933964:80 , srv2112562618:80 , srv799117480:80 , srv1766371739:81 , srv2075192617:81 , srv482084472:81 , srv708518145:81 , srv1545972677:82 , srv316617436:82 , srv384520925:82 , srv484509358:82 , srv622068930:82 , srv1220897953:83 , srv1325246387:83 , srv1334675084:83 , srv1922582811:83 , srv1925602737:83 , srv575329483:83 , srv820935808:83 , srv986285548:83 , srv1885627163:84 , srv212631497:84 , srv325424806:84 , srv355682952:84 , srv476217809:84 , srv898974544:84 , srv913587994:84 , srv1042507277:85 , srv1059702692:85 , srv1109134663:85 , srv1127047112:85 , srv1139330443:85 , srv1234121085:85 , srv1513275022:85 , srv1716540845:85 , srv1951281289:85 , srv1974629959:85 , srv205864601:85 , srv2129429096:85 , srv48373633:85 , srv605047709:85 , srv622294977:85 , srv723689657:85 , srv965888102:85 , srv990370035:85 , srv1019786092:86 , srv1052348479:86 , srv1269622124:86 , srv1367302916:86 , srv1991980814:86 , srv2044260841:86 , srv2061276039:86 , srv246794919:86 , srv275896957:86 , srv372070482:86 , srv718053690:86 , srv739438090:86 , srv1034919098:87 , srv1118297118:87 , srv1189991976:87 , srv1196513703:87 , srv13349597:87 , srv1369938939:87 , srv1578406846:87 , srv2067289538:87 , srv2081287264:87 , srv228596705:87 , srv26151557:87 , srv768924027:87 , srv851560390:87 , srv1306169651:88 , srv1311068106:88 , srv150452741:88 , srv1542664134:88 , srv1693927348:88 , srv1959626483:88 , srv2089368072:88 , srv354540686:88 , srv472766287:88 , srv47951916:88 , srv823371050:88 , srv205788975:89 , srv262687856:89 , srv649499358:89 , srv1445122991:90 , srv1665300419:90 , srv1667970193:90 , srv1717138708:90 , srv1944813605:90 , srv25200564:90 , srv429079866:90 , srv744126112:90 , srv829272143:90 , srv1343818347:91 , srv1805791762:91 , srv41563228:92 , srv2075619821:526 }
2023-09-28 23:53:19,441 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:53:19,443 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv205864601=46, srv316617436=64, srv26151557=61, srv986285548=98, srv2112562618=55, srv1311068106=16, srv1118297118=7, srv1109134663=5, srv1059702692=4, srv1693927348=31, srv1991980814=43, srv1234121085=13, srv25200564=60, srv2075192617=49, srv1325246387=17, srv355682952=67, srv1369938939=22, srv2075619821=50, srv622294977=82, srv575329483=79, srv1052348479=3, srv246794919=59, srv275896957=63, srv820935808=91, srv384520925=69, srv1367302916=21, srv1665300419=29, srv1974629959=42, srv1542664134=26, srv1189991976=10, srv913587994=96, srv2067289538=48, srv1545972677=27, srv1885627163=36, srv2089368072=54, srv1042507277=2, srv372070482=68, srv13349597=19, srv482084472=76, srv476217809=74, srv1220897953=12, srv1717138708=33, srv1716540845=32, srv622068930=81, srv429079866=71, srv1578406846=28, srv1667970193=30, srv1445122991=23, srv1306169651=15, srv1034919098=1, srv262687856=62, srv823371050=92, srv1959626483=41, srv744126112=88, srv472766287=73, srv471078577=72, srv325424806=65, srv1127047112=8, srv723689657=86, srv898974544=95, srv1805791762=35, srv1944813605=39, srv2084933964=53, srv768924027=89, srv990370035=99, srv1922582811=37, srv1019786092=0, srv851560390=94, srv205788975=45, srv2077091875=51, srv2081287264=52, srv1925602737=38, srv1139330443=9, srv212631497=56, srv1343818347=20, srv2061276039=47, srv2129429096=57, srv1111061847=6, srv829272143=93, srv649499358=83, srv1196513703=11, srv2044260841=44, srv718053690=85, srv1766371739=34, srv1269622124=14, srv47951916=75, srv150452741=24, srv1513275022=25, srv48373633=77, srv605047709=80, srv739438090=87, srv1334675084=18, srv228596705=58, srv484509358=78, srv41563228=70, srv799117480=90, srv1951281289=40, srv965888102=97, srv354540686=66, srv708518145=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:53:19,473 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:53:19,474 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:53:19,475 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:53:19,476 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 23:53:19,477 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 23:53:19,478 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 23:53:19,479 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 23:53:19,480 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 23:53:19,481 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 23:53:19,482 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 23:53:19,483 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 23:53:19,484 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 23:53:19,631 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02015960755521166); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03460535889421652, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02015960755521166); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:19,632 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 23:53:19,632 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.014739817749451399, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.02015960755521166); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.03460535889421652, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.02015960755521166); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0115); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.04591666666666667, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 23:53:19,963 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 447 ms to try 5000 different iterations.  Found a solution that moves 994 regions; Going from a computed imbalance of 0.014739817749451399 to a new imbalance of 5.992987179426885E-4. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010423146132940955); MoveCostFunction : (multiplier=7.0, imbalance=0.11044444444444444, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005763774795025093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:19,965 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1805791762:81 , srv1667970193:82 , srv25200564:82 , srv744126112:82 , srv829272143:82 , srv851560390:82 , srv1334675084:83 , srv1343818347:83 , srv1542664134:83 , srv2089368072:83 , srv262687856:83 , srv355682952:83 , srv575329483:83 , srv622294977:83 , srv1196513703:84 , srv1220897953:84 , srv13349597:84 , srv1369938939:84 , srv1944813605:84 , srv1951281289:84 , srv1959626483:84 , srv205864601:84 , srv2067289538:84 , srv246794919:84 , srv372070482:84 , srv429079866:84 , srv898974544:84 , srv965888102:84 , srv1189991976:85 , srv1445122991:85 , srv150452741:85 , srv1925602737:85 , srv1974629959:85 , srv1991980814:85 , srv205788975:85 , srv2077091875:85 , srv228596705:85 , srv26151557:85 , srv275896957:85 , srv325424806:85 , srv354540686:85 , srv41563228:85 , srv47951916:85 , srv48373633:85 , srv723689657:85 , srv768924027:85 , srv799117480:85 , srv823371050:85 , srv986285548:85 , srv990370035:85 , srv1019786092:86 , srv1034919098:86 , srv1042507277:86 , srv1052348479:86 , srv1111061847:86 , srv1118297118:86 , srv1127047112:86 , srv1234121085:86 , srv1269622124:86 , srv1306169651:86 , srv1513275022:86 , srv1578406846:86 , srv1717138708:86 , srv1766371739:86 , srv1885627163:86 , srv2044260841:86 , srv2084933964:86 , srv2112562618:86 , srv212631497:86 , srv2129429096:86 , srv316617436:86 , srv472766287:86 , srv484509358:86 , srv605047709:86 , srv718053690:86 , srv739438090:86 , srv913587994:86 , srv1109134663:87 , srv1139330443:87 , srv1311068106:87 , srv1325246387:87 , srv1367302916:87 , srv1693927348:87 , srv1922582811:87 , srv2061276039:87 , srv2075192617:87 , srv384520925:87 , srv471078577:87 , srv476217809:87 , srv649499358:87 , srv708518145:87 , srv820935808:87 , srv1059702692:88 , srv1545972677:88 , srv1665300419:88 , srv1716540845:88 , srv2081287264:88 , srv482084472:88 , srv622068930:88 , srv2075619821:556 }
2023-09-28 23:53:19,971 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:53:19,974 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv205864601=46, srv316617436=64, srv26151557=61, srv986285548=98, srv2112562618=55, srv1311068106=16, srv1118297118=7, srv1109134663=5, srv1059702692=4, srv1693927348=31, srv1991980814=43, srv1234121085=13, srv25200564=60, srv2075192617=49, srv1325246387=17, srv355682952=67, srv1369938939=22, srv2075619821=50, srv622294977=82, srv575329483=79, srv1052348479=3, srv246794919=59, srv275896957=63, srv820935808=91, srv384520925=69, srv1367302916=21, srv1665300419=29, srv1974629959=42, srv1542664134=26, srv1189991976=10, srv913587994=96, srv2067289538=48, srv1545972677=27, srv1885627163=36, srv2089368072=54, srv1042507277=2, srv372070482=68, srv13349597=19, srv482084472=76, srv476217809=74, srv1220897953=12, srv1717138708=33, srv1716540845=32, srv622068930=81, srv429079866=71, srv1578406846=28, srv1667970193=30, srv1445122991=23, srv1306169651=15, srv1034919098=1, srv262687856=62, srv823371050=92, srv1959626483=41, srv744126112=88, srv472766287=73, srv471078577=72, srv325424806=65, srv1127047112=8, srv723689657=86, srv898974544=95, srv1805791762=35, srv1944813605=39, srv2084933964=53, srv768924027=89, srv990370035=99, srv1922582811=37, srv1019786092=0, srv851560390=94, srv205788975=45, srv2077091875=51, srv2081287264=52, srv1925602737=38, srv1139330443=9, srv212631497=56, srv1343818347=20, srv2061276039=47, srv2129429096=57, srv1111061847=6, srv829272143=93, srv649499358=83, srv1196513703=11, srv2044260841=44, srv718053690=85, srv1766371739=34, srv1269622124=14, srv47951916=75, srv150452741=24, srv1513275022=25, srv48373633=77, srv605047709=80, srv739438090=87, srv1334675084=18, srv228596705=58, srv484509358=78, srv41563228=70, srv799117480=90, srv1951281289=40, srv965888102=97, srv354540686=66, srv708518145=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:53:19,995 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:53:19,996 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:53:19,997 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:53:20,001 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:53:20,002 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:53:20,003 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 23:53:20,004 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 23:53:20,005 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 23:53:20,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 23:53:20,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 23:53:20,006 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 23:53:20,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 23:53:20,010 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 23:53:20,018 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 23:53:20,019 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 23:53:20,020 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 23:53:20,021 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 23:53:20,022 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 23:53:20,175 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010423146132940955); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005763774795025093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:20,176 WARN  [Time-limited test] balancer.StochasticLoadBalancer(497): calculatedMaxSteps:720000000 for loadbalancer's stochastic walk is larger than maxSteps:5000. Hence load balancing may not work well. Setting parameter "hbase.master.balancer.stochastic.runMaxSteps" to true can overcome this issue.(This config change does not require service restart)
2023-09-28 23:53:20,178 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=5.923385811807026E-4, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.005763774795025093); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.010423146132940955); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.005763774795025093); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=2.5E-4); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.00325); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=5000
2023-09-28 23:53:20,628 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 590 ms to try 5000 different iterations.  Found a solution that moves 218 regions; Going from a computed imbalance of 5.923385811807026E-4 to a new imbalance of 1.957190856367016E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.02422222222222222); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:20,629 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1139330443:83 , srv1716540845:83 , srv718053690:83 , srv744126112:83 , srv1059702692:84 , srv1234121085:84 , srv1269622124:84 , srv1311068106:84 , srv1325246387:84 , srv13349597:84 , srv1369938939:84 , srv1513275022:84 , srv1542664134:84 , srv1545972677:84 , srv1578406846:84 , srv1665300419:84 , srv1667970193:84 , srv1693927348:84 , srv1717138708:84 , srv1922582811:84 , srv1944813605:84 , srv1959626483:84 , srv2044260841:84 , srv2081287264:84 , srv2089368072:84 , srv2129429096:84 , srv228596705:84 , srv316617436:84 , srv354540686:84 , srv384520925:84 , srv41563228:84 , srv429079866:84 , srv476217809:84 , srv482084472:84 , srv48373633:84 , srv484509358:84 , srv575329483:84 , srv622068930:84 , srv649499358:84 , srv708518145:84 , srv820935808:84 , srv823371050:84 , srv829272143:84 , srv898974544:84 , srv913587994:84 , srv965888102:84 , srv990370035:84 , srv1019786092:85 , srv1034919098:85 , srv1042507277:85 , srv1052348479:85 , srv1109134663:85 , srv1111061847:85 , srv1118297118:85 , srv1127047112:85 , srv1189991976:85 , srv1196513703:85 , srv1220897953:85 , srv1306169651:85 , srv1334675084:85 , srv1367302916:85 , srv1445122991:85 , srv150452741:85 , srv1766371739:85 , srv1805791762:85 , srv1885627163:85 , srv1925602737:85 , srv1951281289:85 , srv1974629959:85 , srv1991980814:85 , srv205788975:85 , srv205864601:85 , srv2061276039:85 , srv2067289538:85 , srv2075192617:85 , srv2077091875:85 , srv2084933964:85 , srv2112562618:85 , srv212631497:85 , srv246794919:85 , srv25200564:85 , srv26151557:85 , srv262687856:85 , srv275896957:85 , srv325424806:85 , srv355682952:85 , srv372070482:85 , srv471078577:85 , srv472766287:85 , srv605047709:85 , srv723689657:85 , srv739438090:85 , srv768924027:85 , srv799117480:85 , srv851560390:85 , srv986285548:85 , srv1343818347:86 , srv47951916:86 , srv622294977:86 , srv2075619821:633 }
2023-09-28 23:53:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:53:20,633 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv205864601=46, srv316617436=64, srv26151557=61, srv986285548=98, srv2112562618=55, srv1311068106=16, srv1118297118=7, srv1109134663=5, srv1059702692=4, srv1693927348=31, srv1991980814=43, srv1234121085=13, srv25200564=60, srv2075192617=49, srv1325246387=17, srv355682952=67, srv1369938939=22, srv2075619821=50, srv622294977=82, srv575329483=79, srv1052348479=3, srv246794919=59, srv275896957=63, srv820935808=91, srv384520925=69, srv1367302916=21, srv1665300419=29, srv1974629959=42, srv1542664134=26, srv1189991976=10, srv913587994=96, srv2067289538=48, srv1545972677=27, srv1885627163=36, srv2089368072=54, srv1042507277=2, srv372070482=68, srv13349597=19, srv482084472=76, srv476217809=74, srv1220897953=12, srv1717138708=33, srv1716540845=32, srv622068930=81, srv429079866=71, srv1578406846=28, srv1667970193=30, srv1445122991=23, srv1306169651=15, srv1034919098=1, srv262687856=62, srv823371050=92, srv1959626483=41, srv744126112=88, srv472766287=73, srv471078577=72, srv325424806=65, srv1127047112=8, srv723689657=86, srv898974544=95, srv1805791762=35, srv1944813605=39, srv2084933964=53, srv768924027=89, srv990370035=99, srv1922582811=37, srv1019786092=0, srv851560390=94, srv205788975=45, srv2077091875=51, srv2081287264=52, srv1925602737=38, srv1139330443=9, srv212631497=56, srv1343818347=20, srv2061276039=47, srv2129429096=57, srv1111061847=6, srv829272143=93, srv649499358=83, srv1196513703=11, srv2044260841=44, srv718053690=85, srv1766371739=34, srv1269622124=14, srv47951916=75, srv150452741=24, srv1513275022=25, srv48373633=77, srv605047709=80, srv739438090=87, srv1334675084=18, srv228596705=58, srv484509358=78, srv41563228=70, srv799117480=90, srv1951281289=40, srv965888102=97, srv354540686=66, srv708518145=84} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:53:20,644 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:53:20,645 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 30 is on host 30
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 31 is on host 31
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 32 is on host 32
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 33 is on host 33
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 34 is on host 34
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 35 is on host 35
2023-09-28 23:53:20,646 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 36 is on host 36
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 37 is on host 37
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 38 is on host 38
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 39 is on host 39
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 40 is on host 40
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 41 is on host 41
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 42 is on host 42
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 43 is on host 43
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 44 is on host 44
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 45 is on host 45
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 46 is on host 46
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 47 is on host 47
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 48 is on host 48
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 49 is on host 49
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 50 is on host 50
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 51 is on host 51
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 52 is on host 52
2023-09-28 23:53:20,647 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 53 is on host 53
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 54 is on host 54
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 55 is on host 55
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 56 is on host 56
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 57 is on host 57
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 58 is on host 58
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 59 is on host 59
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 60 is on host 60
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 61 is on host 61
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 62 is on host 62
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 63 is on host 63
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 64 is on host 64
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 65 is on host 65
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 66 is on host 66
2023-09-28 23:53:20,648 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 67 is on host 67
2023-09-28 23:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 68 is on host 68
2023-09-28 23:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 69 is on host 69
2023-09-28 23:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 70 is on host 70
2023-09-28 23:53:20,649 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 71 is on host 71
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 72 is on host 72
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 73 is on host 73
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 74 is on host 74
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 75 is on host 75
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 76 is on host 76
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 77 is on host 77
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 78 is on host 78
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 79 is on host 79
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 80 is on host 80
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 81 is on host 81
2023-09-28 23:53:20,650 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 82 is on host 82
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 83 is on host 83
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 84 is on host 84
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 85 is on host 85
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 86 is on host 86
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 87 is on host 87
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 88 is on host 88
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 89 is on host 89
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 90 is on host 90
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 91 is on host 91
2023-09-28 23:53:20,651 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 92 is on host 92
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 93 is on host 93
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 94 is on host 94
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 95 is on host 95
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 96 is on host 96
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 97 is on host 97
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 98 is on host 98
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 99 is on host 99
2023-09-28 23:53:20,652 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=100, number of racks=4
2023-09-28 23:53:20,763 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=1.804543993780202E-5 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=4.178341294946416E-4); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0018951174787165373); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=4.178341294946416E-4); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=8.333333333333333E-5); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:20,763 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1139330443:83 , srv1716540845:83 , srv718053690:83 , srv744126112:83 , srv1059702692:84 , srv1234121085:84 , srv1269622124:84 , srv1311068106:84 , srv1325246387:84 , srv13349597:84 , srv1369938939:84 , srv1513275022:84 , srv1542664134:84 , srv1545972677:84 , srv1578406846:84 , srv1665300419:84 , srv1667970193:84 , srv1693927348:84 , srv1717138708:84 , srv1922582811:84 , srv1944813605:84 , srv1959626483:84 , srv2044260841:84 , srv2081287264:84 , srv2089368072:84 , srv2129429096:84 , srv228596705:84 , srv316617436:84 , srv354540686:84 , srv384520925:84 , srv41563228:84 , srv429079866:84 , srv476217809:84 , srv482084472:84 , srv48373633:84 , srv484509358:84 , srv575329483:84 , srv622068930:84 , srv649499358:84 , srv708518145:84 , srv820935808:84 , srv823371050:84 , srv829272143:84 , srv898974544:84 , srv913587994:84 , srv965888102:84 , srv990370035:84 , srv1019786092:85 , srv1034919098:85 , srv1042507277:85 , srv1052348479:85 , srv1109134663:85 , srv1111061847:85 , srv1118297118:85 , srv1127047112:85 , srv1189991976:85 , srv1196513703:85 , srv1220897953:85 , srv1306169651:85 , srv1334675084:85 , srv1367302916:85 , srv1445122991:85 , srv150452741:85 , srv1766371739:85 , srv1805791762:85 , srv1885627163:85 , srv1925602737:85 , srv1951281289:85 , srv1974629959:85 , srv1991980814:85 , srv205788975:85 , srv205864601:85 , srv2061276039:85 , srv2067289538:85 , srv2075192617:85 , srv2077091875:85 , srv2084933964:85 , srv2112562618:85 , srv212631497:85 , srv246794919:85 , srv25200564:85 , srv26151557:85 , srv262687856:85 , srv275896957:85 , srv325424806:85 , srv355682952:85 , srv372070482:85 , srv471078577:85 , srv472766287:85 , srv605047709:85 , srv723689657:85 , srv739438090:85 , srv768924027:85 , srv799117480:85 , srv851560390:85 , srv986285548:85 , srv1343818347:86 , srv47951916:86 , srv622294977:86 , srv2075619821:633 }
2023-09-28 23:53:20,794 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnLargeClusterWithRacks Thread=12 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=369 (was 340) - SystemLoadAverage LEAK? -, ProcessCount=173 (was 173), AvailableMemoryMB=5939 (was 6554)
2023-09-28 23:53:20,805 INFO  [Time-limited test] hbase.ResourceChecker(147): before: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=12, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=369, ProcessCount=173, AvailableMemoryMB=5939
2023-09-28 23:53:20,806 INFO  [Time-limited test] balancer.BaseLoadBalancer(1082): slop=0.0, systemTablesOnMaster=false
2023-09-28 23:53:20,806 INFO  [Time-limited test] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=5000, runMaxSteps=true, stepsPerRegion=800, maxRunningTime=120000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 111077.0 etc.
2023-09-28 23:53:20,819 INFO  [Time-limited test] balancer.BalancerTestBase(547): Mock Cluster : { srv1095211658:84 , srv1188197716:84 , srv126159090:84 , srv1348494152:84 , srv1383367336:84 , srv140654759:84 , srv1439119830:84 , srv1579235595:84 , srv1579633876:84 , srv1604071817:84 , srv16574432:84 , srv167004772:84 , srv1729192368:84 , srv1733986602:84 , srv1784436707:84 , srv1874559809:84 , srv188368537:84 , srv1959113185:84 , srv1988360626:84 , srv2053853495:84 , srv2091373410:84 , srv324106053:84 , srv343344700:84 , srv357848776:84 , srv644738952:84 , srv744412469:84 , srv801956654:84 , srv95385477:84 , srv972774396:84 , srv1035772385:264 } [srvr=30 rgns=2700 avg=90.0 max=90 min=90]
2023-09-28 23:53:20,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:53:20,821 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1579235595=8, srv1188197716=2, srv1874559809=16, srv343344700=23, srv1959113185=18, srv95385477=28, srv167004772=12, srv1733986602=14, srv188368537=17, srv2053853495=20, srv1035772385=0, srv801956654=27, srv140654759=6, srv1095211658=1, srv1604071817=10, srv1348494152=4, srv2091373410=21, srv744412469=26, srv357848776=24, srv644738952=25, srv16574432=11, srv1784436707=15, srv1988360626=19, srv324106053=22, srv972774396=29, srv1579633876=9, srv1729192368=13, srv126159090=3, srv1383367336=5, srv1439119830=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:53:20,835 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:53:20,836 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:53:20,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:53:20,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:53:20,837 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-28 23:53:20,860 INFO  [Time-limited test] balancer.StochasticLoadBalancer(345): Running balancer because at least one server hosts replicas of the same region. function cost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1043734725296268, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:53:20,860 INFO  [Time-limited test] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.9912521920668114, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06666666666666667, need balance); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=1.1043734725296268, need balance); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=1.0, need balance); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=1.0, need balance); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0);  computedMaxSteps=64800000
2023-09-28 23:55:20,841 INFO  [Time-limited test] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 120001 ms to try 5750715 different iterations.  Found a solution that moves 1861 regions; Going from a computed imbalance of 0.9912521920668114 to a new imbalance of 4.3436668390529226E-5. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.6892592592592592, need balance); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:55:20,848 INFO  [Time-limited test] balancer.BalancerTestBase(563): Mock after balance: { srv1035772385:90 , srv1095211658:90 , srv1188197716:90 , srv126159090:90 , srv1348494152:90 , srv1383367336:90 , srv140654759:90 , srv1439119830:90 , srv1579235595:90 , srv1579633876:90 , srv1604071817:90 , srv16574432:90 , srv167004772:90 , srv1729192368:90 , srv1733986602:90 , srv1784436707:90 , srv1874559809:90 , srv188368537:90 , srv1959113185:90 , srv1988360626:90 , srv2053853495:90 , srv2091373410:90 , srv324106053:90 , srv343344700:90 , srv357848776:90 , srv644738952:90 , srv744412469:90 , srv801956654:90 , srv95385477:90 , srv972774396:90 }
2023-09-28 23:55:20,850 DEBUG [Time-limited test] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster.
2023-09-28 23:55:20,851 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(334): Hosts are {srv1579235595=8, srv1188197716=2, srv1874559809=16, srv343344700=23, srv1959113185=18, srv95385477=28, srv167004772=12, srv1733986602=14, srv188368537=17, srv2053853495=20, srv1035772385=0, srv801956654=27, srv140654759=6, srv1095211658=1, srv1604071817=10, srv1348494152=4, srv2091373410=21, srv744412469=26, srv357848776=24, srv644738952=25, srv16574432=11, srv1784436707=15, srv1988360626=19, srv324106053=22, srv972774396=29, srv1579633876=9, srv1729192368=13, srv126159090=3, srv1383367336=5, srv1439119830=7} racks are {rack_1=1, rack_0=0, rack_3=3, rack_2=2}
2023-09-28 23:55:20,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0
2023-09-28 23:55:20,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 1
2023-09-28 23:55:20,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 2
2023-09-28 23:55:20,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 3
2023-09-28 23:55:20,856 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 4 is on host 4
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 5 is on host 5
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 6 is on host 6
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 7 is on host 7
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 8 is on host 8
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 9 is on host 9
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 10 is on host 10
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 11 is on host 11
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 12 is on host 12
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 13 is on host 13
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 14 is on host 14
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 15 is on host 15
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 16 is on host 16
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 17 is on host 17
2023-09-28 23:55:20,857 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 18 is on host 18
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 19 is on host 19
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 20 is on host 20
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 21 is on host 21
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 22 is on host 22
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 23 is on host 23
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 24 is on host 24
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 25 is on host 25
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 26 is on host 26
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 27 is on host 27
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 28 is on host 28
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(362): server 29 is on host 29
2023-09-28 23:55:20,858 DEBUG [Time-limited test] balancer.BaseLoadBalancer$Cluster(378): Number of tables=10, number of hosts=30, number of racks=4
2023-09-28 23:55:20,870 INFO  [Time-limited test] balancer.StochasticLoadBalancer(381): Cluster wide - skipping load balancing because weighted average imbalance=0.0 <= threshold(0.025). If you want more aggressive balancing, either lower hbase.master.balancer.stochastic.minCostNeedBalance from 0.025 or increase the relative multiplier(s) of the specific cost function(s). functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); MoveCostFunction : (multiplier=7.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.0); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 
2023-09-28 23:55:20,870 INFO  [Time-limited test] balancer.BalancerTestBase(570): Mock Final balance: { srv1035772385:90 , srv1095211658:90 , srv1188197716:90 , srv126159090:90 , srv1348494152:90 , srv1383367336:90 , srv140654759:90 , srv1439119830:90 , srv1579235595:90 , srv1579633876:90 , srv1604071817:90 , srv16574432:90 , srv167004772:90 , srv1729192368:90 , srv1733986602:90 , srv1784436707:90 , srv1874559809:90 , srv188368537:90 , srv1959113185:90 , srv1988360626:90 , srv2053853495:90 , srv2091373410:90 , srv324106053:90 , srv343344700:90 , srv357848776:90 , srv644738952:90 , srv744412469:90 , srv801956654:90 , srv95385477:90 , srv972774396:90 }
2023-09-28 23:55:20,883 INFO  [Time-limited test] hbase.ResourceChecker(175): after: master.balancer.TestStochasticLoadBalancerRegionReplicaWithRacks#testRegionReplicationOnMidClusterWithRacks Thread=11 (was 12), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=231 (was 369), ProcessCount=174 (was 173) - ProcessCount LEAK? -, AvailableMemoryMB=4732 (was 5939)