Skip to main content

Welcome to Apache OpenDAL™

OpenDAL represents Open Data Access Layer. Our vision is to access data freely.

What does OpenDAL do?​

Getting started​

See the page for quick start with multiple languages: Quickstart.

Why OpenDAL?​

The vision of OpenDAL is access data freely, where "free" refers to four essential aspects:

1. Free from services​

OpenDAL must enable users to access various storage services ranging from s3 to dropbox via its own native API. It should provide a unified API for accessing all these services.

For example, we DO

while we DO NOT

  • Add support for Google Cloud Storage (GCS) via XML API: GCS has native JSON API which is more powerful
  • Add support for structural data in MySQL/PostgreSQL: We can treat a database as a simple key-value store, but we can't support unified access of structural data.

2. Free from implementations​

OpenDAL needs to separate the various implementation details of services and enables users to write identical logic for different services.

For example, we DO

  • Add a new capability to indicate whether presign is supported: Users can now write logic based on the can_presign option.
  • Add a default_storage_class configuration for the S3 service: Configuration is specific to the S3 service.
  • Add an option for content_type in the write operation: It aligns with HTTP standards.

while we DO NOT

  • Add a new option in read for storage_class: As different services could have varying values for this parameter.

3. Free to integrate​

OpenDAL needs to be integrated with different systems.

For example, we DO

  • Add Python binding: Python programmers can use OpenDAL.
  • Add object_store integration: object_store users can adopt OpenDAL.

4. Free of cost​

OpenDAL needs to implement features in a zero cost way which means:

  • Users don't need to pay costs for unused features.
  • Users cannot write better implementation for used features.

For example, we DO

  • Add layer support: Users can add logging/metrics/tracing in zero cost way.
  • Implement seek for Reader: Users cannot write better seek support, they all need to pay the same cost.

we DO NOT

  • Add Arc for metadata: Users may only need to use metadata once and never clone it. For those who do want this feature, they can add Arc themselves.