2024-04-03 03:53:17,962 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.region.TestMasterRegionCompaction timeout: 13 mins 2024-04-03 03:53:18,003 INFO [Time-limited test] hbase.ResourceChecker(147): before: master.region.TestMasterRegionCompaction#test Thread=9, OpenFileDescriptor=290, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=170, AvailableMemoryMB=11337 2024-04-03 03:53:18,036 INFO [Time-limited test] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2024-04-03 03:53:18,036 INFO [Time-limited test] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2024-04-03 03:53:18,758 INFO [Time-limited test] region.MasterRegion(309): Create or load local region for table 'test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2024-04-03 03:53:18,869 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2024-04-03 03:53:19,348 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2024-04-03 03:53:19,625 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2024-04-03 03:53:19,626 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2024-04-03 03:53:19,629 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:19,638 DEBUG [Time-limited test] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.createMasterRegion(MasterRegionTestBase.java:109) at org.apache.hadoop.hbase.master.region.MasterRegionTestBase.setUp(MasterRegionTestBase.java:83) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:19,644 INFO [Time-limited test] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2024-04-03 03:53:19,814 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2024-04-03 03:53:19,902 INFO [Time-limited test] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2024-04-03 03:53:19,961 DEBUG [Time-limited test] util.ClassSize(228): Using Unsafe to estimate memory layout 2024-04-03 03:53:19,972 INFO [Time-limited test] regionserver.HRegion(7693): Creating {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='test:local', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'f2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local-tmp 2024-04-03 03:53:20,094 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-04-03 03:53:20,212 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-04-03 03:53:20,212 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:20,213 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:20,214 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-04-03 03:53:20,214 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:20,214 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:20,214 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:20,217 DEBUG [Time-limited test] region.MasterRegion(191): WALDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/WALs/localhost,12345,1712116398727 2024-04-03 03:53:20,263 INFO [Time-limited test] wal.AbstractFSWAL(492): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=localhost%2C12345%2C1712116398727, suffix=, logDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/WALs/localhost,12345,1712116398727, archiveDir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/oldWALs, maxLogs=32 2024-04-03 03:53:20,344 INFO [Time-limited test] wal.AbstractFSWAL(809): New WAL /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/WALs/localhost,12345,1712116398727/localhost%2C12345%2C1712116398727.1712116400281 2024-04-03 03:53:20,345 DEBUG [Time-limited test] wal.AbstractFSWAL(890): Create new AsyncFSWAL writer with pipeline: [] 2024-04-03 03:53:20,346 DEBUG [Time-limited test] regionserver.HRegion(7854): Opening region: {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''} 2024-04-03 03:53:20,347 DEBUG [Time-limited test] regionserver.HRegion(866): Instantiated test:local,,1.fb3251d5e68787855bc0679a918ea02e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2024-04-03 03:53:20,351 DEBUG [Time-limited test] regionserver.HRegion(7894): checking encryption for fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,353 DEBUG [Time-limited test] regionserver.HRegion(7897): checking classloading for fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,437 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,439 WARN [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(529): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1. DEBUG log level might have more details. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:20,440 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:20,477 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f1 2024-04-03 03:53:20,498 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-04-03 03:53:20,499 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f2 of region fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,500 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(532): Unable to set storagePolicy=HOT for path=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2 java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:20,500 DEBUG [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] util.CommonFSUtils(539): The underlying FileSystem implementation doesn't support setStoragePolicy. This is probably intentional on their part, since HDFS-9345 appears to be present in your version of Hadoop. For more information check the Hadoop documentation on 'ArchivalStorage', the Hadoop FileSystem specification docs from HADOOP-11981, and/or related documentation from the provider of the underlying FileSystem (its name should appear in the stacktrace that accompanies this message). Note in particular that Hadoop's local filesystem implementation doesn't support storage policies. java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support setStoragePolicy at org.apache.hadoop.fs.FileSystem.setStoragePolicy(FileSystem.java:3041) at org.apache.hadoop.fs.FilterFileSystem.setStoragePolicy(FilterFileSystem.java:643) at org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:521) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:504) at org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:477) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:220) at org.apache.hadoop.hbase.regionserver.HStore.(HStore.java:252) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:6371) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1117) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1114) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:20,502 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fb3251d5e68787855bc0679a918ea02e columnFamilyName f2 2024-04-03 03:53:20,502 INFO [StoreOpener-fb3251d5e68787855bc0679a918ea02e-1] regionserver.HStore(310): Store=fb3251d5e68787855bc0679a918ea02e/f2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2024-04-03 03:53:20,507 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,508 DEBUG [Time-limited test] regionserver.HRegion(5209): Found 0 recovered edits file(s) under file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,529 DEBUG [Time-limited test] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table test:local descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2024-04-03 03:53:20,533 DEBUG [Time-limited test] regionserver.HRegion(1055): writing seq id for fb3251d5e68787855bc0679a918ea02e 2024-04-03 03:53:20,546 DEBUG [Time-limited test] wal.WALSplitUtil(408): Wrote file=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2024-04-03 03:53:20,547 INFO [Time-limited test] regionserver.HRegion(1072): Opened fb3251d5e68787855bc0679a918ea02e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67268463, jitterRate=0.002378210425376892}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2024-04-03 03:53:20,547 DEBUG [Time-limited test] regionserver.HRegion(965): Region open journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:20,549 INFO [Time-limited test] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2024-04-03 03:53:20,573 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2024-04-03 03:53:20,574 INFO [Time-limited test] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2024-04-03 03:53:20,578 DEBUG [Time-limited test] cleaner.HFileCleaner(242): Starting for large file=Thread[Time-limited test-HFileCleaner.large.0-1712116400576,5,FailOnTimeoutGroup] 2024-04-03 03:53:20,578 DEBUG [Time-limited test] cleaner.HFileCleaner(257): Starting for small files=Thread[Time-limited test-HFileCleaner.small.0-1712116400578,5,FailOnTimeoutGroup] 2024-04-03 03:53:20,578 INFO [Time-limited test] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS is enabled. 2024-04-03 03:53:20,579 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:20,665 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-03 03:53:20,747 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 2cfcfd9ae68942baa1e36aa24510e857 2024-04-03 03:53:20,810 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/2cfcfd9ae68942baa1e36aa24510e857 2024-04-03 03:53:20,888 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on d5355e6401e348dc879a8f792808fb49 2024-04-03 03:53:20,890 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=4 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/d5355e6401e348dc879a8f792808fb49 2024-04-03 03:53:20,894 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/2cfcfd9ae68942baa1e36aa24510e857 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857 2024-04-03 03:53:20,897 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857, entries=1, sequenceid=4, filesize=4.8 K 2024-04-03 03:53:20,899 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/d5355e6401e348dc879a8f792808fb49 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49 2024-04-03 03:53:20,902 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49, entries=1, sequenceid=4, filesize=4.8 K 2024-04-03 03:53:20,904 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 238ms, sequenceid=4, compaction requested=false 2024-04-03 03:53:20,905 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:20,906 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-03 03:53:20,913 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on a1c2078152bd4df38457af878298f708 2024-04-03 03:53:20,916 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/a1c2078152bd4df38457af878298f708 2024-04-03 03:53:20,928 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 890ed067fe1a4108987ce3ab9a3c0bc2 2024-04-03 03:53:20,930 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=8 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/890ed067fe1a4108987ce3ab9a3c0bc2 2024-04-03 03:53:20,933 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/a1c2078152bd4df38457af878298f708 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708 2024-04-03 03:53:20,935 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708, entries=1, sequenceid=8, filesize=4.8 K 2024-04-03 03:53:20,936 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/890ed067fe1a4108987ce3ab9a3c0bc2 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2 2024-04-03 03:53:20,939 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2, entries=1, sequenceid=8, filesize=4.8 K 2024-04-03 03:53:20,940 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 34ms, sequenceid=8, compaction requested=false 2024-04-03 03:53:20,940 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:20,941 INFO [Time-limited test] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=62 B heapSize=752 B 2024-04-03 03:53:20,949 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on 9660bb3905c342cba06f25b43d28232d 2024-04-03 03:53:20,951 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/9660bb3905c342cba06f25b43d28232d 2024-04-03 03:53:20,959 DEBUG [Time-limited test] hfile.HFile$WriterFactory(307): Unable to set drop behind on a89228a80c6f4c21b5e1a13ed5760b13 2024-04-03 03:53:20,961 INFO [Time-limited test] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=12 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/a89228a80c6f4c21b5e1a13ed5760b13 2024-04-03 03:53:20,964 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/9660bb3905c342cba06f25b43d28232d as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d 2024-04-03 03:53:20,966 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d, entries=1, sequenceid=12, filesize=4.8 K 2024-04-03 03:53:20,967 DEBUG [Time-limited test] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/a89228a80c6f4c21b5e1a13ed5760b13 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13 2024-04-03 03:53:20,969 INFO [Time-limited test] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13, entries=1, sequenceid=12, filesize=4.8 K 2024-04-03 03:53:20,970 INFO [Time-limited test] regionserver.HRegion(2948): Finished flush of dataSize ~62 B/62, heapSize ~720 B/720, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 29ms, sequenceid=12, compaction requested=true 2024-04-03 03:53:20,971 DEBUG [Time-limited test] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:20,975 INFO [test:local-Flusher] regionserver.HRegion(2745): Flushing fb3251d5e68787855bc0679a918ea02e 2/2 column families, dataSize=31 B heapSize=632 B 2024-04-03 03:53:20,980 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-04-03 03:53:20,992 DEBUG [test:local-Flusher] hfile.HFile$WriterFactory(307): Unable to set drop behind on 5d259aabbed9485c8f70b1a99c32fef8 2024-04-03 03:53:20,994 INFO [test:local-Flusher] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=31 B at sequenceid=16 (bloomFilter=true), to=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/5d259aabbed9485c8f70b1a99c32fef8 2024-04-03 03:53:20,997 DEBUG [test:local-Flusher] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/5d259aabbed9485c8f70b1a99c32fef8 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8 2024-04-03 03:53:21,000 INFO [test:local-Flusher] regionserver.HStore(1080): Added file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8, entries=1, sequenceid=16, filesize=4.8 K 2024-04-03 03:53:21,001 INFO [test:local-Flusher] regionserver.HRegion(2948): Finished flush of dataSize ~31 B/31, heapSize ~360 B/360, currentSize=0 B/0 for fb3251d5e68787855bc0679a918ea02e in 26ms, sequenceid=16, compaction requested=true 2024-04-03 03:53:21,002 DEBUG [test:local-Flusher] regionserver.HRegion(2446): Flush status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:21,004 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2024-04-03 03:53:21,006 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f1 is initiating major compaction (all files) 2024-04-03 03:53:21,007 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f1 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:21,007 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=19 K 2024-04-03 03:53:21,008 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 2cfcfd9ae68942baa1e36aa24510e857, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1712116400615 2024-04-03 03:53:21,008 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting a1c2078152bd4df38457af878298f708, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1712116400905 2024-04-03 03:53:21,008 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 9660bb3905c342cba06f25b43d28232d, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1712116400940 2024-04-03 03:53:21,008 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 5d259aabbed9485c8f70b1a99c32fef8, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=16, earliestPutTs=1712116400974 2024-04-03 03:53:21,024 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 8e4e8b1857004658a785d684902c6ba2 2024-04-03 03:53:21,030 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f1/8e4e8b1857004658a785d684902c6ba2 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/8e4e8b1857004658a785d684902c6ba2 2024-04-03 03:53:21,040 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 4 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f1 of fb3251d5e68787855bc0679a918ea02e into 8e4e8b1857004658a785d684902c6ba2(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-04-03 03:53:21,041 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:21,041 DEBUG [test:local-Store-Compactor] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2024-04-03 03:53:21,041 DEBUG [test:local-Store-Compactor] regionserver.HStore(1912): fb3251d5e68787855bc0679a918ea02e/f2 is initiating major compaction (all files) 2024-04-03 03:53:21,041 INFO [test:local-Store-Compactor] regionserver.HRegion(2259): Starting compaction of fb3251d5e68787855bc0679a918ea02e/f2 in test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:21,041 INFO [test:local-Store-Compactor] regionserver.HStore(1468): Starting compaction of [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13] into tmpdir=file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp, totalSize=14.3 K 2024-04-03 03:53:21,042 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting d5355e6401e348dc879a8f792808fb49, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=4, earliestPutTs=1712116400615 2024-04-03 03:53:21,042 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting 890ed067fe1a4108987ce3ab9a3c0bc2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=8, earliestPutTs=1712116400905 2024-04-03 03:53:21,043 DEBUG [test:local-Store-Compactor] compactions.Compactor(207): Compacting a89228a80c6f4c21b5e1a13ed5760b13, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=12, earliestPutTs=1712116400940 2024-04-03 03:53:21,052 DEBUG [test:local-Store-Compactor] hfile.HFile$WriterFactory(307): Unable to set drop behind on 4dd416505a5f4c62a439bb8cb3875b79 2024-04-03 03:53:21,056 DEBUG [test:local-Store-Compactor] regionserver.HRegionFileSystem(485): Committing file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/.tmp/f2/4dd416505a5f4c62a439bb8cb3875b79 as file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/4dd416505a5f4c62a439bb8cb3875b79 2024-04-03 03:53:21,060 INFO [test:local-Store-Compactor] regionserver.HStore(1652): Completed major compaction of 3 (all) file(s) in fb3251d5e68787855bc0679a918ea02e/f2 of fb3251d5e68787855bc0679a918ea02e into 4dd416505a5f4c62a439bb8cb3875b79(size=4.9 K), total size for store is 4.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2024-04-03 03:53:21,060 DEBUG [test:local-Store-Compactor] regionserver.HRegion(2289): Compaction status journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:21,060 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8] to archive 2024-04-03 03:53:21,064 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-04-03 03:53:21,067 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857 2024-04-03 03:53:21,067 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708 2024-04-03 03:53:21,068 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d 2024-04-03 03:53:21,069 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8 2024-04-03 03:53:21,080 DEBUG [hfile_cleaner-dir-scan-pool-0] cleaner.CleanerChore(463): Failed to traverse and delete the path: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2024-04-03 03:53:21,080 INFO [TestMasterRegionCompaction.Chore.1] cleaner.CleanerChore(239): Failed to traverse and delete the dir: file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive java.util.concurrent.ExecutionException: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive does not exist at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357) at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.runCleaner(CleanerChore.java:237) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.chore(CleanerChore.java:210) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.FileNotFoundException: File file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive does not exist at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:451) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1804) at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1849) at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:675) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.traverseAndDelete(CleanerChore.java:413) at org.apache.hadoop.hbase.master.cleaner.CleanerChore.lambda$runCleaner$0(CleanerChore.java:236) ... 3 more 2024-04-03 03:53:21,081 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-04-03 03:53:21,081 INFO [Time-limited test] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2024-04-03 03:53:21,095 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/a1c2078152bd4df38457af878298f708$-masterlocalhfile-$ 2024-04-03 03:53:21,095 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/2cfcfd9ae68942baa1e36aa24510e857$-masterlocalhfile-$ 2024-04-03 03:53:21,096 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/9660bb3905c342cba06f25b43d28232d$-masterlocalhfile-$ 2024-04-03 03:53:21,097 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f1/5d259aabbed9485c8f70b1a99c32fef8$-masterlocalhfile-$ 2024-04-03 03:53:21,097 DEBUG [test:local-Store-Compactor] regionserver.HStore(2712): Moving the files [file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13] to archive 2024-04-03 03:53:21,098 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(360): Archiving compacted files. 2024-04-03 03:53:21,099 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49 2024-04-03 03:53:21,100 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2 2024-04-03 03:53:21,101 DEBUG [test:local-Store-Compactor] backup.HFileArchiver(582): Archived from FileableStoreFile, file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13 2024-04-03 03:53:21,103 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/890ed067fe1a4108987ce3ab9a3c0bc2$-masterlocalhfile-$ 2024-04-03 03:53:21,104 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/a89228a80c6f4c21b5e1a13ed5760b13$-masterlocalhfile-$ 2024-04-03 03:53:21,105 INFO [test:local-Store-Compactor] region.MasterRegionUtils(50): Moved file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/local/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49 to file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/f0a39afd-4555-010e-f59b-ed6da4144d0a/archive/data/test/local/fb3251d5e68787855bc0679a918ea02e/f2/d5355e6401e348dc879a8f792808fb49$-masterlocalhfile-$ 2024-04-03 03:53:21,589 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 10 ms. 2024-04-03 03:53:22,083 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-03 03:53:22,583 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-03 03:53:23,082 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-03 03:53:23,582 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-03 03:53:24,082 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-03 03:53:24,582 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-03 03:53:25,082 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 3 ms. 2024-04-03 03:53:25,582 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-04-03 03:53:26,083 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-03 03:53:26,583 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 4 ms. 2024-04-03 03:53:27,081 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-03 03:53:27,580 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-03 03:53:28,080 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-03 03:53:28,580 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-03 03:53:29,080 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 1 ms. 2024-04-03 03:53:29,582 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 2 ms. 2024-04-03 03:53:30,079 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:30,579 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:31,079 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:31,579 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:32,079 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:32,579 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:33,079 DEBUG [TestMasterRegionCompaction.Chore.1] hbase.ScheduledChore(162): HFileCleaner execution time: 0 ms. 2024-04-03 03:53:33,186 INFO [Time-limited test] region.MasterRegion(167): Closing local region {ENCODED => fb3251d5e68787855bc0679a918ea02e, NAME => 'test:local,,1.fb3251d5e68787855bc0679a918ea02e.', STARTKEY => '', ENDKEY => ''}, isAbort=true 2024-04-03 03:53:33,190 DEBUG [Time-limited test] regionserver.HRegion(1604): Closing fb3251d5e68787855bc0679a918ea02e, disabling compactions & flushes 2024-04-03 03:53:33,190 INFO [Time-limited test] regionserver.HRegion(1626): Closing region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:33,190 DEBUG [Time-limited test] regionserver.HRegion(1647): Waiting without time limit for close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:33,190 DEBUG [Time-limited test] regionserver.HRegion(1714): Acquired close lock on test:local,,1.fb3251d5e68787855bc0679a918ea02e. after waiting 0 ms 2024-04-03 03:53:33,190 DEBUG [Time-limited test] regionserver.HRegion(1724): Updates disabled for region test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:33,192 INFO [Time-limited test] regionserver.HRegion(1838): Closed test:local,,1.fb3251d5e68787855bc0679a918ea02e. 2024-04-03 03:53:33,193 DEBUG [Time-limited test] regionserver.HRegion(1558): Region close journal for fb3251d5e68787855bc0679a918ea02e: 2024-04-03 03:53:33,193 INFO [test:local-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2024-04-03 03:53:33,193 INFO [Time-limited test] hbase.ChoreService(369): Chore service for: TestMasterRegionCompaction had [ScheduledChore name=HFileCleaner, period=500, unit=MILLISECONDS] on shutdown 2024-04-03 03:53:33,228 INFO [Time-limited test] hbase.ResourceChecker(175): after: master.region.TestMasterRegionCompaction#test Thread=16 (was 9) Potentially hanging thread: process reaper sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.large.0-1712116400576 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) Potentially hanging thread: org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner java.lang.Object.wait(Native Method) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144) java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165) org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3693) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Time-limited test-HFileCleaner.small.0-1712116400578 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) Potentially hanging thread: HBase-Metrics2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: Monitor thread for TaskMonitor java.lang.Thread.sleep(Native Method) org.apache.hadoop.hbase.monitoring.TaskMonitor$MonitorRunnable.run(TaskMonitor.java:327) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=294 (was 290) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=291 (was 300), ProcessCount=170 (was 170), AvailableMemoryMB=10456 (was 11337)