2023-07-18 08:56:20,101 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.io.hfile.bucket.TestBucketCacheRefCnt timeout: 13 mins 2023-07-18 08:56:20,268 DEBUG [main] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-18 08:56:20,305 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.bucket.TestBucketCacheRefCnt#testMarkStaleAsEvicted Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=123, ProcessCount=170, AvailableMemoryMB=7167 2023-07-18 08:56:20,344 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-07-18 08:56:20,372 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-07-18 08:56:20,393 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=33554432, buckets=1024, bucket capacity=32768=(4*8192)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-07-18 08:56:20,395 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=1 KB, writerThreadNum=1, writerQLen=1000, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-07-18 08:56:21,580 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-07-18 08:56:21,582 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-07-18 08:56:21,591 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.bucket.TestBucketCacheRefCnt#testMarkStaleAsEvicted Thread=10 (was 10), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=123 (was 123), ProcessCount=170 (was 170), AvailableMemoryMB=7105 (was 7167) 2023-07-18 08:56:21,599 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.bucket.TestBucketCacheRefCnt#testBlockInBackingMap Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=123, ProcessCount=170, AvailableMemoryMB=7105 2023-07-18 08:56:21,614 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-07-18 08:56:21,630 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-07-18 08:56:21,632 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=33554432, buckets=1024, bucket capacity=32768=(4*8192)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-07-18 08:56:21,632 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=1 KB, writerThreadNum=1, writerQLen=1000, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-07-18 08:56:22,734 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-07-18 08:56:22,734 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-07-18 08:56:22,743 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.bucket.TestBucketCacheRefCnt#testBlockInBackingMap Thread=10 (was 10), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 123), ProcessCount=170 (was 170), AvailableMemoryMB=7091 (was 7105) 2023-07-18 08:56:22,752 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.bucket.TestBucketCacheRefCnt#testReplacingBlockAndGettingBlockConcurrently Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=170, AvailableMemoryMB=7091 2023-07-18 08:56:22,765 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-07-18 08:56:22,774 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-07-18 08:56:22,776 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=33554432, buckets=1024, bucket capacity=32768=(4*8192)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-07-18 08:56:22,776 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=1 KB, writerThreadNum=1, writerQLen=1000, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-07-18 08:56:24,982 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-07-18 08:56:24,982 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-07-18 08:56:24,991 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.bucket.TestBucketCacheRefCnt#testReplacingBlockAndGettingBlockConcurrently Thread=10 (was 10), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 112), ProcessCount=170 (was 170), AvailableMemoryMB=7531 (was 7091) - AvailableMemoryMB LEAK? - 2023-07-18 08:56:24,999 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.bucket.TestBucketCacheRefCnt#testInBucketCache Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=170, AvailableMemoryMB=7530 2023-07-18 08:56:25,012 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-07-18 08:56:25,027 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-07-18 08:56:25,033 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=33554432, buckets=1024, bucket capacity=32768=(4*8192)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-07-18 08:56:25,034 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=1 KB, writerThreadNum=1, writerQLen=1000, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-07-18 08:56:25,035 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-07-18 08:56:25,036 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-07-18 08:56:25,043 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.bucket.TestBucketCacheRefCnt#testInBucketCache Thread=11 (was 10) Potentially hanging thread: Time-limited test-BucketCacheWriter-0 sun.misc.Unsafe.getIntVolatile(Native Method) sun.misc.Unsafe.getAndAddInt(Unsafe.java:1034) java.util.concurrent.atomic.AtomicInteger.getAndAdd(AtomicInteger.java:177) java.lang.ThreadLocal.nextHashCode(ThreadLocal.java:107) java.lang.ThreadLocal.(ThreadLocal.java:87) java.util.concurrent.locks.ReentrantReadWriteLock$Sync$ThreadLocalHoldCounter.(ReentrantReadWriteLock.java:286) java.util.concurrent.locks.ReentrantReadWriteLock$Sync.(ReentrantReadWriteLock.java:338) java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync.(ReentrantReadWriteLock.java:669) java.util.concurrent.locks.ReentrantReadWriteLock.(ReentrantReadWriteLock.java:240) java.util.concurrent.locks.ReentrantReadWriteLock.(ReentrantReadWriteLock.java:230) org.apache.hadoop.hbase.util.IdReadWriteLock$1.createObject(IdReadWriteLock.java:68) org.apache.hadoop.hbase.util.IdReadWriteLock$1.createObject(IdReadWriteLock.java:65) org.apache.hadoop.hbase.util.ObjectPool.get(ObjectPool.java:162) org.apache.hadoop.hbase.util.IdReadWriteLock.getLock(IdReadWriteLock.java:94) org.apache.hadoop.hbase.io.hfile.bucket.BucketEntry.withWriteLock(BucketEntry.java:213) org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.doDrain(BucketCache.java:1125) org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:973) - Thread LEAK? -, OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 112), ProcessCount=170 (was 170), AvailableMemoryMB=7492 (was 7530) 2023-07-18 08:56:25,052 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.bucket.TestBucketCacheRefCnt#testEvictingBlockCachingBlockGettingBlockConcurrently Thread=10, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=112, ProcessCount=170, AvailableMemoryMB=7491 2023-07-18 08:56:25,063 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-07-18 08:56:25,071 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-07-18 08:56:25,072 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=33554432, buckets=1024, bucket capacity=32768=(4*8192)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-07-18 08:56:25,075 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=1 KB, writerThreadNum=1, writerQLen=1000, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-07-18 08:56:25,080 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-07-18 08:56:25,080 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-07-18 08:56:25,088 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.bucket.TestBucketCacheRefCnt#testEvictingBlockCachingBlockGettingBlockConcurrently Thread=10 (was 10), OpenFileDescriptor=263 (was 263), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=112 (was 112), ProcessCount=170 (was 170), AvailableMemoryMB=7454 (was 7491)