Welcome to Apache OpenDAL
OpenDAL represents Open Data Access Layer. Our vision is to access data freely.
What OpenDAL does?
Getting started
See the page for quick start with multiple languages: Quickstart.
Why OpenDAL?
The vision of OpenDAL is access data freely, where "free" refers to four essential aspects:
1. Free from services
OpenDAL must enable users to access various storage services ranging from s3
to dropbox
via its own native API. It should provide a unified API for accessing all these services.
For example, we DO
- Add support for Google Drive: It allows users to access and manage their data on the Google Drive.
- Add support for Object Storage Service (OSS) via native API: Users can utilize Aliyun's RAM support.
- Add support for supabase storage: Users can visit
supabase storage
now!
while we DO NOT
- Add support for Google Cloud Storage(GCS) via XML API: GCS has native JSON API which more powerful
- Add support for structural data in
MySQL/PostgreSQL
: We can treat a database as a simple key-value store, but we can't support unified access of structural data.
2. Free from implementations
OpenDAL needs to separate the various implementation details of services and enables users to write identical logic for different services.
For example, we DO
- Add a new capability to indicate whether
presign
is supported: Users can now write logic based on thecan_presign
option. - Add a
default_storage_class
configuration for the S3 service: Configuration is specific to the S3 service. - Add an option for
content_type
in thewrite
operation: It aligns with HTTP standards.
while we DO NOT
- Add a new option in read for
storage_class
: As different services could have varying values for this parameter.
3. Free to integrate
OpenDAL needs to be integrated with different systems.
For example, we DO
- Add Python binding: Python programmers can use OpenDAL.
- Add object_store integration:
object_store
users can adopt OpenDAL.
4. Free to zero cost
OpenDAL needs to implement features in zero cost way which means:
- Users need not to pay cost for unused features.
- Users cannot write better implementation for used features.
For example, we DO
- Add
layer
support: Users can add logging/metrics/tracing in zero cost way. - Implement
seek
for Reader: Users cannot write betterseek
support, they all need to pay the same cost.
we DO NOT
- Add
Arc
for metadata: Users may only need to use metadata once and never clone it. For those who do want this feature, they can addArc
themselves.