2024-04-18 13:59:47,443 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.region.TestMasterRegionCompaction timeout: 13 mins 2024-04-18 13:59:47,502 INFO [Time-limited test] hbase.ResourceChecker(147): before: master.region.TestMasterRegionCompaction#test Thread=9, OpenFileDescriptor=289, MaxFileDescriptor=60000, SystemLoadAverage=1306, ProcessCount=212, AvailableMemoryMB=2213 2024-04-18 13:59:47,552 INFO [Time-limited test] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2024-04-18 13:59:47,552 INFO [Time-limited test] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2024-04-18 13:59:48,356 INFO [Time-limited test] region.MasterRegion(309): Create or load local region for table 'test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2024-04-18 13:59:48,547 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-04-18 13:59:49,148 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2024-04-18 13:59:49,404 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2024-04-18 13:59:49,404 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2024-04-18 13:59:49,407 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:49,420 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:49,428 INFO [Time-limited test] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2024-04-18 13:59:49,661 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2024-04-18 13:59:49,778 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2024-04-18 13:59:49,863 DEBUG [Time-limited test] util.ClassSize(228): Using Unsafe to estimate memory layout 2024-04-18 13:59:49,877 INFO [Time-limited test] regionserver.HRegion(7693): Creating {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local-tmp 2024-04-18 13:59:50,039 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-04-18 13:59:50,177 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-04-18 13:59:50,177 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 13:59:50,177 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 13:59:50,178 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-04-18 13:59:50,179 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 13:59:50,179 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 13:59:50,179 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:50,182 DEBUG [Time-limited test] region.MasterRegion(191): WALDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/WALs/localhost,12345,1713448788308 2024-04-18 13:59:50,229 INFO [Time-limited test] wal.AbstractFSWAL(492): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=localhost%2C12345%2C1713448788308, suffix=, logDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/WALs/localhost,12345,1713448788308, archiveDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/oldWALs, maxLogs=32 2024-04-18 13:59:50,368 INFO [Time-limited test] wal.AbstractFSWAL(809): New WAL /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/WALs/localhost,12345,1713448788308/localhost%2C12345%2C1713448788308.1713448790251 2024-04-18 13:59:50,369 DEBUG [Time-limited test] wal.AbstractFSWAL(890): Create new AsyncFSWAL writer with pipeline: [] 2024-04-18 13:59:50,370 DEBUG [Time-limited test] regionserver.HRegion(7854): Opening region: {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''} 2024-04-18 13:59:50,371 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-04-18 13:59:50,387 DEBUG [Time-limited test] regionserver.HRegion(7894): checking encryption for fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,389 DEBUG [Time-limited test] regionserver.HRegion(7897): checking classloading for fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,482 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,486 WARN [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(529): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1. DEBUG log level might have more details. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:50,487 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:50,536 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f1 2024-04-18 13:59:50,565 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-04-18 13:59:50,570 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f2 of region fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,571 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(532): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2 java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:50,572 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:50,574 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f2 2024-04-18 13:59:50,575 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-04-18 13:59:50,582 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,587 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,609 DEBUG [Time-limited test] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table test:local descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2024-04-18 13:59:50,615 DEBUG [Time-limited test] regionserver.HRegion(1055): writing seq id for fb3251d5e68787855bc0679a918ea02e 2024-04-18 13:59:50,656 DEBUG [Time-limited test] wal.WALSplitUtil(408): Wrote file=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2024-04-18 13:59:50,657 INFO [Time-limited test] regionserver.HRegion(1072): Opened fb3251d5e68787855bc0679a918ea02e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67275185, jitterRate=0.002478376030921936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2024-04-18 13:59:50,657 DEBUG [Time-limited test] regionserver.HRegion(965): Region open journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:50,658 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2024-04-18 13:59:50,677 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2024-04-18 13:59:50,678 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2024-04-18 13:59:50,682 DEBUG [Time-limited test] cleaner.HFileCleaner(242): Starting for large file=Thread[Time-limited test-HFileCleaner.large.0-1713448790680,5,FailOnTimeoutGroup] 2024-04-18 13:59:50,682 DEBUG [Time-limited test] cleaner.HFileCleaner(257): Starting for small files=Thread[Time-limited test-HFileCleaner.small.0-1713448790682,5,FailOnTimeoutGroup] 2024-04-18 13:59:50,683 INFO [Time-limited test] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS is enabled. 2024-04-18 13:59:50,687 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 13:59:50,763 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-18 13:59:50,841 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on ab2388ab601f41f4b5266f3203aa634d 2024-04-18 13:59:50,906 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/ab2388ab601f41f4b5266f3203aa634d 2024-04-18 13:59:50,981 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 55e1616fb6b948f6b60bbb70dc23d124 2024-04-18 13:59:50,984 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/55e1616fb6b948f6b60bbb70dc23d124 2024-04-18 13:59:50,986 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/ab2388ab601f41f4b5266f3203aa634d as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d 2024-04-18 13:59:50,990 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d, entries=1, sequenceid=4, filesize=4.8 K 2024-04-18 13:59:50,992 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/55e1616fb6b948f6b60bbb70dc23d124 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124 2024-04-18 13:59:50,995 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124, entries=1, sequenceid=4, filesize=4.8 K 2024-04-18 13:59:50,996 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 233ms, sequenceid=4, compaction requested=false 2024-04-18 13:59:50,998 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:51,001 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-18 13:59:51,044 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 20890fe37bf146bfb2b6b6ca08555cf3 2024-04-18 13:59:51,047 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/20890fe37bf146bfb2b6b6ca08555cf3 2024-04-18 13:59:51,057 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 7b44d55cff5540d6b1b6933c94b678ee 2024-04-18 13:59:51,060 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/7b44d55cff5540d6b1b6933c94b678ee 2024-04-18 13:59:51,062 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/20890fe37bf146bfb2b6b6ca08555cf3 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3 2024-04-18 13:59:51,065 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3, entries=1, sequenceid=8, filesize=4.8 K 2024-04-18 13:59:51,071 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/7b44d55cff5540d6b1b6933c94b678ee as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee 2024-04-18 13:59:51,074 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee, entries=1, sequenceid=8, filesize=4.8 K 2024-04-18 13:59:51,076 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 75ms, sequenceid=8, compaction requested=false 2024-04-18 13:59:51,077 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:51,078 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-18 13:59:51,100 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 0083aa8f4769431fbe25fdbb8bcdca1d 2024-04-18 13:59:51,102 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/0083aa8f4769431fbe25fdbb8bcdca1d 2024-04-18 13:59:51,131 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on f1b59a0d06cd4945895af92e559e9de3 2024-04-18 13:59:51,133 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/f1b59a0d06cd4945895af92e559e9de3 2024-04-18 13:59:51,136 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/0083aa8f4769431fbe25fdbb8bcdca1d as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d 2024-04-18 13:59:51,139 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d, entries=1, sequenceid=12, filesize=4.8 K 2024-04-18 13:59:51,140 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/f1b59a0d06cd4945895af92e559e9de3 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3 2024-04-18 13:59:51,143 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3, entries=1, sequenceid=12, filesize=4.8 K 2024-04-18 13:59:51,144 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 66ms, sequenceid=12, compaction requested=true 2024-04-18 13:59:51,144 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:51,150 INFO [test:local-Flusher] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=31 B heapSize=632 B 2024-04-18 13:59:51,154 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-04-18 13:59:51,177 DEBUG [test:local-Flusher] hfile.HFile$WriterFactory(307): Unable to set drop behind on 90ab4792741f4757b7c348f981b0a507 2024-04-18 13:59:51,179 INFO [test:local-Flusher] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=16 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/90ab4792741f4757b7c348f981b0a507 2024-04-18 13:59:51,181 DEBUG [test:local-Flusher] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/90ab4792741f4757b7c348f981b0a507 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507 2024-04-18 13:59:51,184 INFO [test:local-Flusher] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507, entries=1, sequenceid=16, filesize=4.8 K 2024-04-18 13:59:51,185 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(463): Failed to traverse and delete the path: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-18 13:59:51,185 INFO [test:local-Flusher] regionserver.HRegion(2948): Finished flush of dataSize ~31 B/31, heapSize ~360 B/360, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 35ms, sequenceid=16, compaction requested=true 2024-04-18 13:59:51,185 INFO [TestMasterRegionCompaction.Chore.1] cleaner.CleanerChore(239): Failed to traverse and delete the dir: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive does not exist at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.runCleaner(CleanerChore.java:237) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:210) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) ... 3 more 2024-04-18 13:59:51,186 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-04-18 13:59:51,185 DEBUG [test:local-Flusher] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:51,189 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2024-04-18 13:59:51,192 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f1 is initiating major compaction (all files) 2024-04-18 13:59:51,192 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f1 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 13:59:51,193 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=19 K 2024-04-18 13:59:51,194 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting ab2388ab601f41f4b5266f3203aa634d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1713448790716 2024-04-18 13:59:51,194 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 20890fe37bf146bfb2b6b6ca08555cf3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1713448790998 2024-04-18 13:59:51,194 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 0083aa8f4769431fbe25fdbb8bcdca1d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1713448791077 2024-04-18 13:59:51,195 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 90ab4792741f4757b7c348f981b0a507, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=16, earliestPutTs=1713448791148 2024-04-18 13:59:51,221 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 41b6828a23fb448d8b3a6f7a21a49784 2024-04-18 13:59:51,229 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/41b6828a23fb448d8b3a6f7a21a49784 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/41b6828a23fb448d8b3a6f7a21a49784 2024-04-18 13:59:51,245 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 4 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f1 of fb3251d5e68787855bc0679a918ea02e into 41b6828a23fb448d8b3a6f7a21a49784(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-04-18 13:59:51,245 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:51,246 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2024-04-18 13:59:51,246 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f2 is initiating major compaction (all files) 2024-04-18 13:59:51,246 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f2 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 13:59:51,246 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=14.3 K 2024-04-18 13:59:51,247 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 55e1616fb6b948f6b60bbb70dc23d124, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1713448790716 2024-04-18 13:59:51,247 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 7b44d55cff5540d6b1b6933c94b678ee, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1713448790998 2024-04-18 13:59:51,247 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting f1b59a0d06cd4945895af92e559e9de3, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1713448791077 2024-04-18 13:59:51,262 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 31b41b523bf4495db64ae3fa2af0b55f 2024-04-18 13:59:51,266 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/31b41b523bf4495db64ae3fa2af0b55f as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/31b41b523bf4495db64ae3fa2af0b55f 2024-04-18 13:59:51,270 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 3 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f2 of fb3251d5e68787855bc0679a918ea02e into 31b41b523bf4495db64ae3fa2af0b55f(size=4.9 K), total size for store is 4.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-04-18 13:59:51,270 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 13:59:51,271 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507] to archive 2024-04-18 13:59:51,274 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-04-18 13:59:51,276 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d 2024-04-18 13:59:51,277 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3 2024-04-18 13:59:51,278 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d 2024-04-18 13:59:51,279 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507 2024-04-18 13:59:51,305 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/90ab4792741f4757b7c348f981b0a507$-masterlocalhfile-$ 2024-04-18 13:59:51,307 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/0083aa8f4769431fbe25fdbb8bcdca1d$-masterlocalhfile-$ 2024-04-18 13:59:51,308 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/ab2388ab601f41f4b5266f3203aa634d$-masterlocalhfile-$ 2024-04-18 13:59:51,309 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/20890fe37bf146bfb2b6b6ca08555cf3$-masterlocalhfile-$ 2024-04-18 13:59:51,315 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3] to archive 2024-04-18 13:59:51,316 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-04-18 13:59:51,317 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124 2024-04-18 13:59:51,318 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee 2024-04-18 13:59:51,319 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3 2024-04-18 13:59:51,332 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/7b44d55cff5540d6b1b6933c94b678ee$-masterlocalhfile-$ 2024-04-18 13:59:51,332 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/f1b59a0d06cd4945895af92e559e9de3$-masterlocalhfile-$ 2024-04-18 13:59:51,333 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/e6c67d86-3abe-5fd8-20cd-9ac0bcb2dbd9/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/55e1616fb6b948f6b60bbb70dc23d124$-masterlocalhfile-$ 2024-04-18 13:59:51,355 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-04-18 13:59:51,696 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 13 ms. 2024-04-18 13:59:52,190 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 7 ms. 2024-04-18 13:59:52,687 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-18 13:59:53,187 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-18 13:59:53,687 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-18 13:59:54,188 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-18 13:59:54,687 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-18 13:59:55,187 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-18 13:59:55,688 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-18 13:59:56,188 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-18 13:59:56,787 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 104 ms. 2024-04-18 13:59:57,185 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-18 13:59:57,685 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-18 13:59:58,185 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-18 13:59:58,686 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-04-18 13:59:59,185 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-18 13:59:59,686 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-18 14:00:00,183 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:00,683 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:01,183 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:01,683 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:02,183 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:02,683 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:03,183 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-18 14:00:03,361 INFO [Time-limited test] region.MasterRegion(167): Closing local region {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, isAbort=true 2024-04-18 14:00:03,366 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-04-18 14:00:03,366 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 14:00:03,366 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 14:00:03,366 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-04-18 14:00:03,366 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 14:00:03,369 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-18 14:00:03,370 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-18 14:00:03,370 INFO [test:local-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2024-04-18 14:00:03,370 INFO [Time-limited test] hbase.ChoreService(369): Chore service for: TestMasterRegionCompaction had [ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS] on shutdown 2024-04-18 14:00:03,418 INFO [Time-limited test] hbase.ResourceChecker(175): after: master.region.TestMasterRegionCompaction#test Thread=16 (was 9) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.small.0-1713448790682 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.large.0-1713448790680 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) - Thread LEAK? -, OpenFileDescriptor=293 (was 289) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=1161 (was 1306), ProcessCount=215 (was 212) - ProcessCount LEAK? -, AvailableMemoryMB=1662 (was 2213)