5Azure Blob Storage

6Google Cloud Storage

7Oracle Cloud Infrastructure (OCI) Object Storage

8Local Storage

9Guides

10Repository

Micronaut Object Storage

Micronaut Object Storage provides a uniform API to create, read and delete objects in the major cloud providers

Version: 2.7.0

1 Introduction

Micronaut Object Storage provides a uniform API to create, read and delete objects in the major cloud providers:

There is also a local storage implementation for testing purposes.

Using this API enables the creation of truly multi-cloud, portable applications.

2 Release History

For this project, you can find a list of releases (with release notes) here:

https://github.com/micronaut-projects/micronaut-object-storage/releases

3 Quick Start

To get started, you need to declare a dependency for the actual cloud provider you are using. See the actual cloud provider documentation for more details:

Amazon S3.
Azure Blob Storage.
Google Cloud Storage.
Oracle Cloud Infrastructure (OCI) Object Storage.

Then, you can inject in your controllers/services/etc. a bean of type ObjectStorageOperations, the parent interface that allows you to use the API in a generic way for all cloud providers:

JavaGroovyKotlin

@Singleton
public class ProfileService {

    private static final Logger LOG = LoggerFactory.getLogger(ProfileService.class);

    private final ObjectStorageOperations<?, ?, ?> objectStorage;

    public ProfileService(ObjectStorageOperations<?, ?, ?> objectStorage) {
        this.objectStorage = objectStorage;
    }

}

Copy to Clipboard

@Singleton
class ProfileService {

    final ObjectStorageOperations<?, ?, ?> objectStorage

    ProfileService(ObjectStorageOperations<?, ?, ?> objectStorage) {
        this.objectStorage = objectStorage
    }

}

Copy to Clipboard

If your application is not multi-cloud, and/or you need cloud-specific details, you can use a concrete implementation. For example, for AWS S3:

JavaGroovyKotlin

@Controller
public class UploadController {

    private final AwsS3Operations objectStorage;

    public UploadController(AwsS3Operations objectStorage) {
        this.objectStorage = objectStorage;
    }

}

Copy to Clipboard

@Controller
class UploadController {

    final AwsS3Operations objectStorage

    UploadController(AwsS3Operations objectStorage) {
        this.objectStorage = objectStorage
    }

}

Copy to Clipboard

If you have multiple object storages configured, it is possible to select which one to work with via bean qualifiers.

For example, given the following configuration:

src/main/resources/application-ec2.yml

PropertiesYamlTomlGroovyHoconJSON

micronaut:
  object-storage:
    aws:
      pictures:
        bucket: pictures-bucket
      logos:
        bucket: logos-bucket

Copy to Clipboard

[micronaut]
  [micronaut.object-storage]
    [micronaut.object-storage.aws]
      [micronaut.object-storage.aws.pictures]
        bucket="pictures-bucket"
      [micronaut.object-storage.aws.logos]
        bucket="logos-bucket"

Copy to Clipboard

{
  micronaut {
    object-storage {
      aws {
        pictures {
          bucket = "pictures-bucket"
        }
        logos {
          bucket = "logos-bucket"
        }
      }
    }
  }
}

Copy to Clipboard

{
  "micronaut": {
    "object-storage": {
      "aws": {
        "pictures": {
          "bucket": "pictures-bucket"
        },
        "logos": {
          "bucket": "logos-bucket"
        }
      }
    }
  }
}

Copy to Clipboard

You then need to use @Named("pictures") or @Named("logos") to specify which of the object storages you want to use.

Uploading files

JavaGroovyKotlin

public String saveProfilePicture(String userId, Path path) {
    UploadRequest request = UploadRequest.fromPath(path, userId); // (1)
    UploadResponse<?> response = objectStorage.upload(request); // (2)
    return response.getKey(); // (3)
}

Copy to Clipboard

String saveProfilePicture(String userId, Path path) {
    UploadRequest request = UploadRequest.fromPath(path, userId) // (1)
    UploadResponse response = objectStorage.upload(request) // (2)
    response.key // (3)
}

Copy to Clipboard

open fun saveProfilePicture(userId: String, path: Path): String? {
    val request = UploadRequest.fromPath(path, userId) // (1)
    val response = objectStorage.upload(request) // (2)
    return response.key // (3)
}

Copy to Clipboard

1	You can use any of the UploadRequest static methods to build an upload request.
2	The upload operation returns an UploadResponse, which wraps the cloud-specific SDK response object.
3	The response object contains some common properties for all cloud vendor, and a `getNativeResponse()` method that can be used for accessing the vendor-specific response object.

In case you want to have better control of the upload options used, you can use the method upload(UploadRequest, Consumer) of ObjectStorageOperations, which will give you access to the cloud vendor-specific request class or builder.

For example, for AWS S3:

JavaGroovyKotlin

UploadResponse<PutObjectResponse> response = objectStorage.upload(objectStorageUpload, builder -> {
    builder.acl(ObjectCannedACL.PUBLIC_READ);
});

Copy to Clipboard

Retrieving files

JavaGroovyKotlin

public Optional<Path> retrieveProfilePicture(String userId, String fileName) {
    Path destination = null;
    try {
        String key = userId + "/" + fileName;
        Optional<InputStream> stream = objectStorage.retrieve(key) // (1)
            .map(ObjectStorageEntry::getInputStream);
        if (stream.isPresent()) {
            destination = File.createTempFile(userId, "temp").toPath();
            Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING);
            return Optional.of(destination);
        } else {
            return Optional.empty();
        }
    } catch (IOException e) {
        LOG.error("Error while trying to save profile picture to the local file [{}]: {}", destination, e.getMessage());
        return Optional.empty();
    }
}

Copy to Clipboard

Optional<Path> retrieveProfilePicture(String userId, String fileName) {
    String key = "${userId}/${fileName}"
    Optional<InputStream> stream = objectStorage.retrieve(key) // (1)
            .map(ObjectStorageEntry::getInputStream)
    if (stream.isPresent()) {
        Path destination = File.createTempFile(userId, "temp").toPath()
        Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING)
        return Optional.of(destination)
    } else {
        return Optional.empty()
    }
}

Copy to Clipboard

open fun retrieveProfilePicture(userId: String, fileName: String): Path? {
    val key = "$userId/$fileName"
    val stream = objectStorage.retrieve<ObjectStorageEntry<*>>(key) // (1)
        .map { obj: ObjectStorageEntry<*> -> obj.inputStream }

    return if (stream.isPresent) {
        val destination = File.createTempFile(userId, "temp").toPath()
        Files.copy(stream.get(), destination, StandardCopyOption.REPLACE_EXISTING)
        destination
    } else {
        null
    }
}

Copy to Clipboard

1	The retrieve operation returns an ObjectStorageEntry, from which you can get an `InputStream`. There is also a `getNativeEntry()` method that gives you access to the cloud vendor-specific response object.

Deleting files

JavaGroovyKotlin

public void deleteProfilePicture(String userId, String fileName) {
    String key = userId + "/" + fileName;
    objectStorage.delete(key); // (1)
}

Copy to Clipboard

1	The delete operation returns the cloud vendor-specific delete response object in case you need it.

4 Amazon S3

To use Amazon S3, you need the following dependency:

GradleMaven

implementation("io.micronaut.objectstorage:micronaut-object-storage-aws")

Copy to Clipboard

Refer to the Micronaut AWS documentation for more information about credentials and region configuration.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties for AwsS3Configuration
Property	Type	Description
`micronaut.object-storage.aws.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.aws.*.bucket`	java.lang.String	The name of the AWS S3 bucket.

For example:

src/main/resources/application-ec2.yml

PropertiesYamlTomlGroovyHoconJSON

micronaut:
  object-storage:
    aws:
      default:
        bucket: profile-pictures-bucket

Copy to Clipboard

The concrete implementation of ObjectStorageOperations is AwsS3Operations.

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:

@Singleton
public class S3ClientBuilderCustomizer implements BeanCreatedEventListener<S3ClientBuilder> {

    @Override
    public S3ClientBuilder onCreated(@NonNull BeanCreatedEvent<S3ClientBuilder> event) {
        return event.getBean()
            .overrideConfiguration(c -> c.apiCallTimeout(Duration.of(60, ChronoUnit.SECONDS)));
    }
}

See the guide for Use the Micronaut Object Storage API to Store Files in Amazon S3 to learn more.

5 Azure Blob Storage

To use Azure Blob Storage, you need the following dependency:

GradleMaven

implementation("io.micronaut.objectstorage:micronaut-object-storage-azure")

Copy to Clipboard

Refer to the Micronaut Azure documentation for more information about authentication options.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties for AzureBlobStorageConfiguration
Property	Type	Description
`micronaut.object-storage.azure.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.azure.*.container`	java.lang.String	The blob container name.
`micronaut.object-storage.azure.*.endpoint`	java.lang.String	The blob service endpoint, in the format of https://{accountName}.blob.core.windows.net.

For example:

src/main/resources/application-azure.yml

PropertiesYamlTomlGroovyHoconJSON

azure.credential.client-secret.client-id=<client-id>
azure.credential.client-secret.tenant-id=<tenant-id>
azure.credential.client-secret.secret=<secret>
micronaut.object-storage.azure.default.container=profile-pictures-container
micronaut.object-storage.azure.default.endpoint=https://my-account.blob.core.windows.net

Copy to Clipboard

azure:
  credential:
    client-secret:
      client-id: <client-id>
      tenant-id: <tenant-id>
      secret: <secret>

micronaut:
  object-storage:
    azure:
      default:
        container: profile-pictures-container
        endpoint: https://my-account.blob.core.windows.net

Copy to Clipboard

[azure]
  [azure.credential]
    [azure.credential.client-secret]
      client-id="<client-id>"
      tenant-id="<tenant-id>"
      secret="<secret>"
[micronaut]
  [micronaut.object-storage]
    [micronaut.object-storage.azure]
      [micronaut.object-storage.azure.default]
        container="profile-pictures-container"
        endpoint="https://my-account.blob.core.windows.net"

Copy to Clipboard

azure {
  credential {
    clientSecret {
      clientId = "<client-id>"
      tenantId = "<tenant-id>"
      secret = "<secret>"
    }
  }
}
micronaut {
  objectStorage {
    azure {
      'default' {
        container = "profile-pictures-container"
        endpoint = "https://my-account.blob.core.windows.net"
      }
    }
  }
}

Copy to Clipboard

{
  azure {
    credential {
      client-secret {
        client-id = "<client-id>"
        tenant-id = "<tenant-id>"
        secret = "<secret>"
      }
    }
  }
  micronaut {
    object-storage {
      azure {
        default {
          container = "profile-pictures-container"
          endpoint = "https://my-account.blob.core.windows.net"
        }
      }
    }
  }
}

Copy to Clipboard

{
  "azure": {
    "credential": {
      "client-secret": {
        "client-id": "<client-id>",
        "tenant-id": "<tenant-id>",
        "secret": "<secret>"
      }
    }
  },
  "micronaut": {
    "object-storage": {
      "azure": {
        "default": {
          "container": "profile-pictures-container",
          "endpoint": "https://my-account.blob.core.windows.net"
        }
      }
    }
  }
}

Copy to Clipboard

The concrete implementation of ObjectStorageOperations is AzureBlobStorageOperations.

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:

@Singleton
public class BlobServiceClientBuilderCustomizer implements BeanCreatedEventListener<BlobServiceClientBuilder> {

    @Override
    public BlobServiceClientBuilder onCreated(@NonNull BeanCreatedEvent<BlobServiceClientBuilder> event) {
        HttpPipelinePolicy noOp = (context, next) -> next.process();
        return event.getBean().addPolicy(noOp);
    }
}

6 Google Cloud Storage

To use Google Cloud Storage, you need the following dependency:

GradleMaven

implementation("io.micronaut.objectstorage:micronaut-object-storage-gcp")

Copy to Clipboard

Refer to the Micronaut GCP documentation for more information about configuring your GCP project.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties for GoogleCloudStorageConfiguration
Property	Type	Description
`micronaut.object-storage.gcp.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.gcp.*.bucket`	java.lang.String	The name of the Google Cloud Storage bucket.

For example:

src/main/resources/application-gcp.yml

PropertiesYamlTomlGroovyHoconJSON

gcp:
  project-id: my-gcp-project

micronaut:
  object-storage:
    gcp:
      default:
        bucket: profile-pictures-bucket

Copy to Clipboard

[gcp]
  project-id="my-gcp-project"
[micronaut]
  [micronaut.object-storage]
    [micronaut.object-storage.gcp]
      [micronaut.object-storage.gcp.default]
        bucket="profile-pictures-bucket"

Copy to Clipboard

{
  gcp {
    project-id = "my-gcp-project"
  }
  micronaut {
    object-storage {
      gcp {
        default {
          bucket = "profile-pictures-bucket"
        }
      }
    }
  }
}

Copy to Clipboard

{
  "gcp": {
    "project-id": "my-gcp-project"
  },
  "micronaut": {
    "object-storage": {
      "gcp": {
        "default": {
          "bucket": "profile-pictures-bucket"
        }
      }
    }
  }
}

Copy to Clipboard

The concrete implementation of ObjectStorageOperations is GoogleCloudStorageOperations.

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:

@Singleton
public class StorageOptionsBuilderCustomizer implements BeanCreatedEventListener<StorageOptions.Builder> {

    @Override
    public StorageOptions.Builder onCreated(@NonNull BeanCreatedEvent<StorageOptions.Builder> event) {
        return event.getBean()
            .setTransportOptions(HttpTransportOptions.newBuilder().setConnectTimeout(60_000).build());
    }
}

See the guide for Use the Micronaut Object Storage API to Store Files in Google Cloud Storage to learn more.

7 Oracle Cloud Infrastructure (OCI) Object Storage

To use Oracle Cloud Infrastructure (OCI) Object Storage, you need the following dependency:

GradleMaven

implementation("io.micronaut.objectstorage:micronaut-object-storage-oracle-cloud")

Copy to Clipboard

Refer to the Micronaut Oracle Cloud documentation for more information about authentication options.

The object storage specific configuration options available are:

🔗

Table 1. Configuration Properties for OracleCloudStorageConfiguration
Property	Type	Description
`micronaut.object-storage.oracle-cloud.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.oracle-cloud.*.bucket`	java.lang.String	The name of the OCI Object Storage bucket.
`micronaut.object-storage.oracle-cloud.*.namespace`	java.lang.String	The OCI Object Storage namespace used.

For example:

src/main/resources/application-oraclecloud.yml

PropertiesYamlTomlGroovyHoconJSON

oci:
  config:
    profile: DEFAULT

micronaut:
  object-storage:
    oracle-cloud:
      default:
        bucket: profile-pictures-bucket
        namespace: MyNamespace

Copy to Clipboard

[oci]
  [oci.config]
    profile="DEFAULT"
[micronaut]
  [micronaut.object-storage]
    [micronaut.object-storage.oracle-cloud]
      [micronaut.object-storage.oracle-cloud.default]
        bucket="profile-pictures-bucket"
        namespace="MyNamespace"

Copy to Clipboard

oci {
  config {
    profile = "DEFAULT"
  }
}
micronaut {
  objectStorage {
    oracleCloud {
      'default' {
        bucket = "profile-pictures-bucket"
        namespace = "MyNamespace"
      }
    }
  }
}

Copy to Clipboard

{
  oci {
    config {
      profile = "DEFAULT"
    }
  }
  micronaut {
    object-storage {
      oracle-cloud {
        default {
          bucket = "profile-pictures-bucket"
          namespace = "MyNamespace"
        }
      }
    }
  }
}

Copy to Clipboard

{
  "oci": {
    "config": {
      "profile": "DEFAULT"
    }
  },
  "micronaut": {
    "object-storage": {
      "oracle-cloud": {
        "default": {
          "bucket": "profile-pictures-bucket",
          "namespace": "MyNamespace"
        }
      }
    }
  }
}

Copy to Clipboard

The concrete implementation of ObjectStorageOperations is OracleCloudStorageOperations

Advanced configuration

For configuration properties other than the specified above, you can add bean to your application that implements BeanCreatedEventListener. For example:

//See https://github.com/oracle/oci-java-sdk/blob/master/bmc-examples/src/main/java/ClientConfigurationTimeoutExample.java
@Singleton
public class ObjectStorageClientBuilderCustomizer implements BeanCreatedEventListener<ObjectStorageClient.Builder> {

    public static final int CONNECTION_TIMEOUT_IN_MILLISECONDS = 25000;
    public static final int READ_TIMEOUT_IN_MILLISECONDS = 35000;

    @Override
    public ObjectStorageClient.Builder onCreated(@NonNull BeanCreatedEvent<ObjectStorageClient.Builder> event) {
        ClientConfiguration clientConfiguration =
            ClientConfiguration.builder()
                .connectionTimeoutMillis(CONNECTION_TIMEOUT_IN_MILLISECONDS)
                .readTimeoutMillis(READ_TIMEOUT_IN_MILLISECONDS)
                .build();


        return event.getBean()
            .configuration(clientConfiguration);
    }
}

See the guide for Use the Micronaut Object Storage API to Store Files in Oracle Cloud Infrastructure (OCI) Object Storage to learn more.

8 Local Storage

To use the local storage implementation (useful for tests), you need the following dependency:

GradleMaven

testImplementation("io.micronaut.objectstorage:micronaut-object-storage-local")

Copy to Clipboard

Then, simply define a local storage:

PropertiesYamlTomlGroovyHoconJSON

micronaut:
  object-storage:
    local:
      default:
        enabled: true

Copy to Clipboard

When added to the classpath, LocalStorageOperations becomes the primary implementation of ObjectStorageOperations.

By default, it will create a temporary folder to store the files, but you can configure it to use a specific folder:

🔗

Table 1. Configuration Properties for LocalStorageConfiguration
Property	Type	Description
`micronaut.object-storage.local.*.enabled`	boolean	Whether to enable or disable this object storage.
`micronaut.object-storage.local.*.path`	java.nio.file.Path	The path of the local storage.

For example:

src/main/resources/application-test.yml

PropertiesYamlTomlGroovyHoconJSON

micronaut:
  object-storage:
    local:
      default:
        path: /tmp/my-object-storage

Copy to Clipboard

The concrete implementation of ObjectStorageOperations is LocalStorageOperations.

9 Guides

See the following list of guides to learn more about working with Object Storage in the Micronaut Framework:

https://guides.micronaut.io/latest/tag-object_storage.html

10 Repository

You can find the source code of this project in this repository:

https://github.com/micronaut-projects/micronaut-object-storage

Table of Contents

Micronaut Object Storage

1 Introduction

2 Release History

3 Quick Start

Uploading files

Retrieving files

Deleting files

4 Amazon S3

Advanced configuration

5 Azure Blob Storage

Advanced configuration

6 Google Cloud Storage

Advanced configuration

7 Oracle Cloud Infrastructure (OCI) Object Storage

Advanced configuration

8 Local Storage

9 Guides

10 Repository