2023-09-26 08:56:16,788 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.io.hfile.TestHFile timeout: 13 mins 2023-09-26 08:56:17,082 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/4a43d8d0-6eff-c496-8483-10c5a79e9dd9 2023-09-26 08:56:17,238 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-09-26 08:56:17,636 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-09-26 08:56:17,657 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testEmptyHFile Thread=11, OpenFileDescriptor=263, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=167, AvailableMemoryMB=5151 2023-09-26 08:56:17,676 DEBUG [Time-limited test] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-09-26 08:56:17,768 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-09-26 08:56:17,841 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-09-26 08:56:17,874 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on testEmptyHFile 2023-09-26 08:56:18,088 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testEmptyHFile Thread=13 (was 11) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) - Thread LEAK? -, OpenFileDescriptor=264 (was 263) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=351 (was 351), ProcessCount=167 (was 167), AvailableMemoryMB=5111 (was 5151) 2023-09-26 08:56:18,096 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testReaderWithoutBlockCache Thread=13, OpenFileDescriptor=264, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=167, AvailableMemoryMB=5111 2023-09-26 08:56:18,120 WARN [Time-limited test] io.ByteBuffAllocator(181): The config keys hbase.ipc.server.reservoir.initial.buffer.size and hbase.ipc.server.reservoir.initial.max are deprecated now, instead please use hbase.server.allocator.buffer.size and hbase.server.allocator.max.buffer.count. In future release we will remove the two deprecated configs. 2023-09-26 08:56:18,142 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on e106389779004e4f84f2652d8de94cba 2023-09-26 08:56:18,219 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testReaderWithoutBlockCache Thread=13 (was 13), OpenFileDescriptor=265 (was 264) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=351 (was 351), ProcessCount=167 (was 167), AvailableMemoryMB=5109 (was 5111) 2023-09-26 08:56:18,227 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testReaderWithAdaptiveLruCombinedBlockCache Thread=13, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=167, AvailableMemoryMB=5109 2023-09-26 08:56:18,244 WARN [Time-limited test] io.ByteBuffAllocator(181): The config keys hbase.ipc.server.reservoir.initial.buffer.size and hbase.ipc.server.reservoir.initial.max are deprecated now, instead please use hbase.server.allocator.buffer.size and hbase.server.allocator.max.buffer.count. In future release we will remove the two deprecated configs. 2023-09-26 08:56:18,258 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on ba8ff4f0be77417eab7e02f638c9b9f9 2023-09-26 08:56:18,282 INFO [Time-limited test] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-26 08:56:18,298 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-09-26 08:56:18,307 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-09-26 08:56:18,323 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=31518720, buckets=15, bucket capacity=2101248=(4*525312)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-09-26 08:56:18,325 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=64 KB, writerThreadNum=3, writerQLen=64, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-09-26 08:56:18,345 INFO [Time-limited test-BucketCacheWriter-2] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-2 exiting, cacheEnabled=false 2023-09-26 08:56:18,345 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-09-26 08:56:18,345 INFO [Time-limited test-BucketCacheWriter-1] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-1 exiting, cacheEnabled=false 2023-09-26 08:56:18,345 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-09-26 08:56:18,357 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testReaderWithAdaptiveLruCombinedBlockCache Thread=13 (was 13), OpenFileDescriptor=265 (was 265), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=351 (was 351), ProcessCount=167 (was 167), AvailableMemoryMB=5087 (was 5109) 2023-09-26 08:56:18,364 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testCorruptOutOfOrderHFileWrite Thread=13, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=351, ProcessCount=168, AvailableMemoryMB=5086 2023-09-26 08:56:18,937 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testCorruptOutOfOrderHFileWrite Thread=13 (was 13), OpenFileDescriptor=265 (was 265), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 351), ProcessCount=167 (was 168), AvailableMemoryMB=5067 (was 5086) 2023-09-26 08:56:18,947 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testCorruptTruncatedHFile Thread=13, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5066 2023-09-26 08:56:18,951 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on testCorruptTruncatedHFile 2023-09-26 08:56:18,981 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testCorruptTruncatedHFile Thread=13 (was 13), OpenFileDescriptor=265 (was 265), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5062 (was 5066) 2023-09-26 08:56:18,989 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testReaderWithCombinedBlockCache Thread=13, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5062 2023-09-26 08:56:19,004 WARN [Time-limited test] io.ByteBuffAllocator(181): The config keys hbase.ipc.server.reservoir.initial.buffer.size and hbase.ipc.server.reservoir.initial.max are deprecated now, instead please use hbase.server.allocator.buffer.size and hbase.server.allocator.max.buffer.count. In future release we will remove the two deprecated configs. 2023-09-26 08:56:19,011 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 7e35ba1e68504042b858fbac3c0c6666 2023-09-26 08:56:19,033 INFO [Time-limited test] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-26 08:56:19,039 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-09-26 08:56:19,047 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-09-26 08:56:19,047 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=31518720, buckets=15, bucket capacity=2101248=(4*525312)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-09-26 08:56:19,051 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=64 KB, writerThreadNum=3, writerQLen=64, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-09-26 08:56:19,056 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-09-26 08:56:19,056 INFO [Time-limited test-BucketCacheWriter-2] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-2 exiting, cacheEnabled=false 2023-09-26 08:56:19,056 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-09-26 08:56:19,056 INFO [Time-limited test-BucketCacheWriter-1] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-1 exiting, cacheEnabled=false 2023-09-26 08:56:19,067 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testReaderWithCombinedBlockCache Thread=13 (was 13), OpenFileDescriptor=265 (was 265), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5054 (was 5062) 2023-09-26 08:56:19,077 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testCorrupt0LengthHFile Thread=13, OpenFileDescriptor=265, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5054 2023-09-26 08:56:19,087 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testCorrupt0LengthHFile Thread=13 (was 13), OpenFileDescriptor=266 (was 265) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5053 (was 5054) 2023-09-26 08:56:19,096 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testReaderWithLRUBlockCache Thread=13, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5053 2023-09-26 08:56:19,109 WARN [Time-limited test] io.ByteBuffAllocator(181): The config keys hbase.ipc.server.reservoir.initial.buffer.size and hbase.ipc.server.reservoir.initial.max are deprecated now, instead please use hbase.server.allocator.buffer.size and hbase.server.allocator.max.buffer.count. In future release we will remove the two deprecated configs. 2023-09-26 08:56:19,116 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on c46ecd61782a4e47847ce5a4cac1b6f3 2023-09-26 08:56:19,144 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testReaderWithLRUBlockCache Thread=13 (was 13), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5050 (was 5053) 2023-09-26 08:56:19,152 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testMetaBlocks Thread=13, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5049 2023-09-26 08:56:19,216 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testMetaBlocks Thread=13 (was 13), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5047 (was 5049) 2023-09-26 08:56:19,226 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testGetShortMidpoint Thread=13, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5046 2023-09-26 08:56:19,243 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testGetShortMidpoint Thread=13 (was 13), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5045 (was 5046) 2023-09-26 08:56:19,252 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testCompressionOrdinance Thread=13, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5046 2023-09-26 08:56:19,260 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testCompressionOrdinance Thread=13 (was 13), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5045 (was 5046) 2023-09-26 08:56:19,269 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testReaderWithTinyLfuCombinedBlockCache Thread=13, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5045 2023-09-26 08:56:19,289 WARN [Time-limited test] io.ByteBuffAllocator(181): The config keys hbase.ipc.server.reservoir.initial.buffer.size and hbase.ipc.server.reservoir.initial.max are deprecated now, instead please use hbase.server.allocator.buffer.size and hbase.server.allocator.max.buffer.count. In future release we will remove the two deprecated configs. 2023-09-26 08:56:19,294 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on fa8f439134d14fef9d25eb412c449a2b 2023-09-26 08:56:19,319 INFO [Time-limited test] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-26 08:56:19,367 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-09-26 08:56:19,380 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-09-26 08:56:19,381 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=31518720, buckets=15, bucket capacity=2101248=(4*525312)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-09-26 08:56:19,383 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=64 KB, writerThreadNum=3, writerQLen=64, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-09-26 08:56:19,401 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-09-26 08:56:19,401 INFO [Time-limited test-BucketCacheWriter-1] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-1 exiting, cacheEnabled=false 2023-09-26 08:56:19,401 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-09-26 08:56:19,401 INFO [Time-limited test-BucketCacheWriter-2] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-2 exiting, cacheEnabled=false 2023-09-26 08:56:19,410 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testReaderWithTinyLfuCombinedBlockCache Thread=14 (was 13) Potentially hanging thread: ForkJoinPool.commonPool-worker-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.ForkJoinPool.awaitWork(ForkJoinPool.java:1824) java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1693) java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175) - Thread LEAK? -, OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5017 (was 5045) 2023-09-26 08:56:19,418 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testTFileFeatures Thread=14, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5017 2023-09-26 08:56:19,420 INFO [Time-limited test] hfile.TestHFile(498): writer=null, name=org.apache.hadoop.fs.FSDataOutputStream@1fb0fe00, compression=none cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 0 2023-09-26 08:56:19,439 INFO [Time-limited test] hfile.TestHFile(498): writer=null, name=org.apache.hadoop.fs.FSDataOutputStream@501975c2, compression=gz cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 0 2023-09-26 08:56:19,451 INFO [Time-limited test] hfile.TestHFile(498): writer=null, name=org.apache.hadoop.fs.FSDataOutputStream@6aa897af, compression=none cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 0 2023-09-26 08:56:19,462 INFO [Time-limited test] hfile.TestHFile(498): writer=null, name=org.apache.hadoop.fs.FSDataOutputStream@23dc539a, compression=gz cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false 0 2023-09-26 08:56:19,479 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testTFileFeatures Thread=14 (was 14), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5020 (was 5017) - AvailableMemoryMB LEAK? - 2023-09-26 08:56:19,486 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testDBEShipped Thread=14, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5019 2023-09-26 08:56:19,492 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on testDBEShipped_PREFIX 2023-09-26 08:56:19,498 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on testDBEShipped_DIFF 2023-09-26 08:56:19,502 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on testDBEShipped_FAST_DIFF 2023-09-26 08:56:19,506 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on testDBEShipped_ROW_INDEX_V1 2023-09-26 08:56:19,515 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testDBEShipped Thread=14 (was 14), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5018 (was 5019) 2023-09-26 08:56:19,523 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testShortMidpointSameQual Thread=14, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5018 2023-09-26 08:56:19,531 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testShortMidpointSameQual Thread=14 (was 14), OpenFileDescriptor=266 (was 266), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5017 (was 5018) 2023-09-26 08:56:19,539 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testNullMetaBlocks Thread=14, OpenFileDescriptor=266, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5017 2023-09-26 08:56:19,563 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testNullMetaBlocks Thread=14 (was 14), OpenFileDescriptor=268 (was 266) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=5016 (was 5017) 2023-09-26 08:56:19,573 INFO [Time-limited test] hbase.ResourceChecker(147): before: io.hfile.TestHFile#testReaderWithLruCombinedBlockCache Thread=14, OpenFileDescriptor=268, MaxFileDescriptor=60000, SystemLoadAverage=339, ProcessCount=167, AvailableMemoryMB=5016 2023-09-26 08:56:19,588 WARN [Time-limited test] io.ByteBuffAllocator(181): The config keys hbase.ipc.server.reservoir.initial.buffer.size and hbase.ipc.server.reservoir.initial.max are deprecated now, instead please use hbase.server.allocator.buffer.size and hbase.server.allocator.max.buffer.count. In future release we will remove the two deprecated configs. 2023-09-26 08:56:19,592 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on fea56e4fe5544fd78def2e5b1ec55283 2023-09-26 08:56:19,611 INFO [Time-limited test] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-09-26 08:56:19,613 INFO [Time-limited test] util.ByteBufferArray(64): Allocating buffers total=32 MB, sizePerBuffer=2 MB, count=16 2023-09-26 08:56:19,624 INFO [Time-limited test] bucket.BucketCache(278): Instantiating BucketCache with acceptableFactor: 0.95, minFactor: 0.85, extraFreeFactor: 0.1, singleFactor: 0.25, multiFactor: 0.5, memoryFactor: 0.25 2023-09-26 08:56:19,625 INFO [Time-limited test] bucket.BucketAllocator(339): Cache totalSize=31518720, buckets=15, bucket capacity=2101248=(4*525312)=(FEWEST_ITEMS_IN_BUCKET*(largest configured bucketcache size)) 2023-09-26 08:56:19,627 INFO [Time-limited test] bucket.BucketCache(320): Started bucket cache; ioengine=offheap, capacity=32 MB, blockSize=64 KB, writerThreadNum=3, writerQLen=64, persistencePath=null, bucketAllocator=org.apache.hadoop.hbase.io.hfile.bucket.BucketAllocator 2023-09-26 08:56:19,631 ERROR [Time-limited test-BucketCacheWriter-2] bucket.BucketCache$WriterThread(975): WriterThread encountered error java.lang.IllegalArgumentException: ReferenceCount : 0 (expected: > 0) at org.apache.hbase.thirdparty.io.netty.util.internal.ObjectUtil.checkPositive(ObjectUtil.java:100) at org.apache.hadoop.hbase.nio.ByteBuff.checkRefCount(ByteBuff.java:77) at org.apache.hadoop.hbase.nio.MultiByteBuff.duplicate(MultiByteBuff.java:683) at org.apache.hadoop.hbase.nio.MultiByteBuff.duplicate(MultiByteBuff.java:43) at org.apache.hadoop.hbase.io.hfile.HFileBlock.getBufferReadOnly(HFileBlock.java:506) at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$RAMQueueEntry.writeToCache(BucketCache.java:1503) at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.doDrain(BucketCache.java:1059) at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:973) 2023-09-26 08:56:19,634 INFO [Time-limited test-BucketCacheWriter-2] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-2 exiting, cacheEnabled=false 2023-09-26 08:56:19,634 INFO [Time-limited test] bucket.BucketCache(1318): Shutdown bucket cache: IO persistent=false; path to write=null 2023-09-26 08:56:19,636 INFO [Time-limited test-BucketCacheWriter-0] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-0 exiting, cacheEnabled=false 2023-09-26 08:56:19,637 INFO [Time-limited test-BucketCacheWriter-1] bucket.BucketCache$WriterThread(981): Time-limited test-BucketCacheWriter-1 exiting, cacheEnabled=false 2023-09-26 08:56:19,659 INFO [Time-limited test] hbase.ResourceChecker(175): after: io.hfile.TestHFile#testReaderWithLruCombinedBlockCache Thread=16 (was 14) Potentially hanging thread: Time-limited test-BucketCacheWriter-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403) org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getRAMQueueEntries(BucketCache.java:1154) org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:967) Potentially hanging thread: Time-limited test-BucketCacheWriter-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:403) org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getRAMQueueEntries(BucketCache.java:1154) org.apache.hadoop.hbase.io.hfile.bucket.BucketCache$WriterThread.run(BucketCache.java:967) - Thread LEAK? -, OpenFileDescriptor=268 (was 268), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=339 (was 339), ProcessCount=167 (was 167), AvailableMemoryMB=4981 (was 5016)