2024-03-14 14:54:30,138 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.region.TestMasterRegionCompaction timeout: 13 mins 2024-03-14 14:54:30,185 INFO [Time-limited test] hbase.ResourceChecker(147): before: master.region.TestMasterRegionCompaction#test Thread=9, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=170, AvailableMemoryMB=9643 2024-03-14 14:54:30,229 INFO [Time-limited test] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2024-03-14 14:54:30,230 INFO [Time-limited test] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2024-03-14 14:54:30,853 INFO [Time-limited test] region.MasterRegion(309): Create or load local region for table 'test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2024-03-14 14:54:30,963 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-03-14 14:54:31,380 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2024-03-14 14:54:31,604 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2024-03-14 14:54:31,605 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2024-03-14 14:54:31,608 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-03-14 14:54:31,614 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-03-14 14:54:31,620 INFO [Time-limited test] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2024-03-14 14:54:31,797 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2024-03-14 14:54:31,909 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2024-03-14 14:54:31,973 DEBUG [Time-limited test] util.ClassSize(228): Using Unsafe to estimate memory layout 2024-03-14 14:54:31,983 INFO [Time-limited test] regionserver.HRegion(7693): Creating {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local-tmp 2024-03-14 14:54:32,116 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-03-14 14:54:32,237 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-03-14 14:54:32,237 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:32,237 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:32,239 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 1 ms 2024-03-14 14:54:32,239 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:32,239 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:32,239 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,242 DEBUG [Time-limited test] region.MasterRegion(191): WALDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/WALs/localhost,12345,1710428070822 2024-03-14 14:54:32,277 INFO [Time-limited test] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=localhost%2C12345%2C1710428070822, suffix=, logDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/WALs/localhost,12345,1710428070822, archiveDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/oldWALs, maxLogs=32 2024-03-14 14:54:32,379 INFO [Time-limited test] wal.AbstractFSWAL(806): New WAL /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/WALs/localhost,12345,1710428070822/localhost%2C12345%2C1710428070822.1710428072294 2024-03-14 14:54:32,379 DEBUG [Time-limited test] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [] 2024-03-14 14:54:32,380 DEBUG [Time-limited test] regionserver.HRegion(7854): Opening region: {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''} 2024-03-14 14:54:32,382 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-03-14 14:54:32,386 DEBUG [Time-limited test] regionserver.HRegion(7894): checking encryption for fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,389 DEBUG [Time-limited test] regionserver.HRegion(7897): checking classloading for fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,499 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,501 WARN [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(529): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1. DEBUG log level might have more details. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-14 14:54:32,501 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-14 14:54:32,536 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f1 2024-03-14 14:54:32,553 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-03-14 14:54:32,554 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f2 of region fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,555 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(532): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2 java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-14 14:54:32,555 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-03-14 14:54:32,557 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f2 2024-03-14 14:54:32,557 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-03-14 14:54:32,561 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,561 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,575 DEBUG [Time-limited test] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table test:local descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2024-03-14 14:54:32,578 DEBUG [Time-limited test] regionserver.HRegion(1055): writing seq id for fb3251d5e68787855bc0679a918ea02e 2024-03-14 14:54:32,588 DEBUG [Time-limited test] wal.WALSplitUtil(408): Wrote file=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2024-03-14 14:54:32,589 INFO [Time-limited test] regionserver.HRegion(1072): Opened fb3251d5e68787855bc0679a918ea02e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61736532, jitterRate=-0.08005398511886597}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2024-03-14 14:54:32,589 DEBUG [Time-limited test] regionserver.HRegion(965): Region open journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,591 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2024-03-14 14:54:32,610 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2024-03-14 14:54:32,610 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2024-03-14 14:54:32,613 DEBUG [Time-limited test] cleaner.HFileCleaner(242): Starting for large file=Thread[Time-limited test-HFileCleaner.large.0-1710428072613,5,FailOnTimeoutGroup] 2024-03-14 14:54:32,617 DEBUG [Time-limited test] cleaner.HFileCleaner(257): Starting for small files=Thread[Time-limited test-HFileCleaner.small.0-1710428072614,5,FailOnTimeoutGroup] 2024-03-14 14:54:32,617 INFO [Time-limited test] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS is enabled. 2024-03-14 14:54:32,618 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:32,683 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-03-14 14:54:32,741 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 4b88af356ea24621933bb508cc6e1dd2 2024-03-14 14:54:32,788 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/4b88af356ea24621933bb508cc6e1dd2 2024-03-14 14:54:32,843 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on e57486afa28f4f4798f919d74ad9809e 2024-03-14 14:54:32,846 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/e57486afa28f4f4798f919d74ad9809e 2024-03-14 14:54:32,850 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/4b88af356ea24621933bb508cc6e1dd2 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2 2024-03-14 14:54:32,852 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2, entries=1, sequenceid=4, filesize=4.8 K 2024-03-14 14:54:32,853 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/e57486afa28f4f4798f919d74ad9809e as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e 2024-03-14 14:54:32,856 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e, entries=1, sequenceid=4, filesize=4.8 K 2024-03-14 14:54:32,857 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 174ms, sequenceid=4, compaction requested=false 2024-03-14 14:54:32,858 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,859 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-03-14 14:54:32,868 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on ec3571d1b49046b991c11e834dbd8acf 2024-03-14 14:54:32,870 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/ec3571d1b49046b991c11e834dbd8acf 2024-03-14 14:54:32,879 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on adbe6a0e25a34d79b70970f6d0b01c51 2024-03-14 14:54:32,881 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/adbe6a0e25a34d79b70970f6d0b01c51 2024-03-14 14:54:32,883 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/ec3571d1b49046b991c11e834dbd8acf as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf 2024-03-14 14:54:32,885 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf, entries=1, sequenceid=8, filesize=4.8 K 2024-03-14 14:54:32,886 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/adbe6a0e25a34d79b70970f6d0b01c51 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51 2024-03-14 14:54:32,889 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51, entries=1, sequenceid=8, filesize=4.8 K 2024-03-14 14:54:32,890 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 31ms, sequenceid=8, compaction requested=false 2024-03-14 14:54:32,890 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,891 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-03-14 14:54:32,901 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 3e3e9c6968da46878b0b551a94f84410 2024-03-14 14:54:32,904 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/3e3e9c6968da46878b0b551a94f84410 2024-03-14 14:54:32,912 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 9be6c386b48049acaf8fe79d11ac20c5 2024-03-14 14:54:32,914 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/9be6c386b48049acaf8fe79d11ac20c5 2024-03-14 14:54:32,916 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/3e3e9c6968da46878b0b551a94f84410 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410 2024-03-14 14:54:32,918 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410, entries=1, sequenceid=12, filesize=4.8 K 2024-03-14 14:54:32,918 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/9be6c386b48049acaf8fe79d11ac20c5 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5 2024-03-14 14:54:32,920 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5, entries=1, sequenceid=12, filesize=4.8 K 2024-03-14 14:54:32,921 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 30ms, sequenceid=12, compaction requested=true 2024-03-14 14:54:32,921 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,924 INFO [test:local-Flusher] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=31 B heapSize=632 B 2024-03-14 14:54:32,926 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-03-14 14:54:32,930 DEBUG [test:local-Flusher] hfile.HFile$WriterFactory(307): Unable to set drop behind on da5652ef57fa4a4ba7eeca9b5b47472e 2024-03-14 14:54:32,931 INFO [test:local-Flusher] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=16 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/da5652ef57fa4a4ba7eeca9b5b47472e 2024-03-14 14:54:32,933 DEBUG [test:local-Flusher] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/da5652ef57fa4a4ba7eeca9b5b47472e as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e 2024-03-14 14:54:32,935 INFO [test:local-Flusher] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e, entries=1, sequenceid=16, filesize=4.8 K 2024-03-14 14:54:32,936 INFO [test:local-Flusher] regionserver.HRegion(2948): Finished flush of dataSize ~31 B/31, heapSize ~360 B/360, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 12ms, sequenceid=16, compaction requested=true 2024-03-14 14:54:32,936 DEBUG [test:local-Flusher] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,938 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2024-03-14 14:54:32,940 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f1 is initiating major compaction (all files) 2024-03-14 14:54:32,940 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f1 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:32,940 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=19 K 2024-03-14 14:54:32,941 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 4b88af356ea24621933bb508cc6e1dd2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1710428072650 2024-03-14 14:54:32,941 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting ec3571d1b49046b991c11e834dbd8acf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1710428072859 2024-03-14 14:54:32,942 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 3e3e9c6968da46878b0b551a94f84410, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1710428072890 2024-03-14 14:54:32,942 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting da5652ef57fa4a4ba7eeca9b5b47472e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=16, earliestPutTs=1710428072923 2024-03-14 14:54:32,957 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on d9fa3e518d0142a29f9cad74d0cffcea 2024-03-14 14:54:32,962 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/d9fa3e518d0142a29f9cad74d0cffcea as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/d9fa3e518d0142a29f9cad74d0cffcea 2024-03-14 14:54:32,971 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 4 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f1 of fb3251d5e68787855bc0679a918ea02e into d9fa3e518d0142a29f9cad74d0cffcea(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-03-14 14:54:32,971 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,971 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2024-03-14 14:54:32,971 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f2 is initiating major compaction (all files) 2024-03-14 14:54:32,971 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f2 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:32,971 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=14.3 K 2024-03-14 14:54:32,972 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting e57486afa28f4f4798f919d74ad9809e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1710428072650 2024-03-14 14:54:32,972 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting adbe6a0e25a34d79b70970f6d0b01c51, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1710428072859 2024-03-14 14:54:32,972 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 9be6c386b48049acaf8fe79d11ac20c5, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1710428072890 2024-03-14 14:54:32,979 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 507648f873fa43409f66475f25e36e2d 2024-03-14 14:54:32,982 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/507648f873fa43409f66475f25e36e2d as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/507648f873fa43409f66475f25e36e2d 2024-03-14 14:54:32,984 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 3 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f2 of fb3251d5e68787855bc0679a918ea02e into 507648f873fa43409f66475f25e36e2d(size=4.9 K), total size for store is 4.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-03-14 14:54:32,984 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:32,985 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e] to archive 2024-03-14 14:54:32,988 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-03-14 14:54:32,990 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2 2024-03-14 14:54:32,990 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf 2024-03-14 14:54:32,991 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410 2024-03-14 14:54:32,992 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e 2024-03-14 14:54:33,011 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/4b88af356ea24621933bb508cc6e1dd2$-masterlocalhfile-$ 2024-03-14 14:54:33,012 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ec3571d1b49046b991c11e834dbd8acf$-masterlocalhfile-$ 2024-03-14 14:54:33,012 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/da5652ef57fa4a4ba7eeca9b5b47472e$-masterlocalhfile-$ 2024-03-14 14:54:33,013 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/3e3e9c6968da46878b0b551a94f84410$-masterlocalhfile-$ 2024-03-14 14:54:33,013 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5] to archive 2024-03-14 14:54:33,013 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-03-14 14:54:33,014 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e 2024-03-14 14:54:33,015 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51 2024-03-14 14:54:33,015 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5 2024-03-14 14:54:33,017 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/e57486afa28f4f4798f919d74ad9809e$-masterlocalhfile-$ 2024-03-14 14:54:33,018 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/adbe6a0e25a34d79b70970f6d0b01c51$-masterlocalhfile-$ 2024-03-14 14:54:33,018 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/824df348-2733-ce74-2a92-df988013f049/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/9be6c386b48049acaf8fe79d11ac20c5$-masterlocalhfile-$ 2024-03-14 14:54:33,027 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-03-14 14:54:33,128 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 10 ms. 2024-03-14 14:54:33,622 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-03-14 14:54:34,121 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-14 14:54:34,621 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-14 14:54:35,121 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-14 14:54:35,621 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-14 14:54:36,121 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-03-14 14:54:36,621 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-14 14:54:37,121 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-14 14:54:37,621 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-14 14:54:38,123 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 5 ms. 2024-03-14 14:54:38,619 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-03-14 14:54:39,183 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 65 ms. 2024-03-14 14:54:39,619 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-03-14 14:54:40,119 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-03-14 14:54:40,619 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-03-14 14:54:41,120 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-03-14 14:54:41,618 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:42,118 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:42,618 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:43,118 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:43,618 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:44,118 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:44,618 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-03-14 14:54:45,031 INFO [Time-limited test] region.MasterRegion(167): Closing local region {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, isAbort=true 2024-03-14 14:54:45,036 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-03-14 14:54:45,036 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:45,036 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:45,036 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-03-14 14:54:45,036 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:45,039 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-03-14 14:54:45,039 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-03-14 14:54:45,039 INFO [test:local-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2024-03-14 14:54:45,039 INFO [Time-limited test] hbase.ChoreService(369): Chore service for: TestMasterRegionCompaction had [ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS] on shutdown 2024-03-14 14:54:45,080 INFO [Time-limited test] hbase.ResourceChecker(175): after: master.region.TestMasterRegionCompaction#test Thread=16 (was 9) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.large.0-1710428072613 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.small.0-1710428072614 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 290) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=220 (was 250), ProcessCount=168 (was 170), AvailableMemoryMB=8771 (was 9643)