Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate Libs To Rely on get_file_contents Rather Than Presigned S3 URLs #10419

Closed
indiv0 opened this issue Jul 1, 2024 · 0 comments · Fixed by #10456
Closed

Migrate Libs To Rely on get_file_contents Rather Than Presigned S3 URLs #10419

indiv0 opened this issue Jul 1, 2024 · 0 comments · Fixed by #10456
Assignees
Labels
-libs Libraries: New libraries to be implemented p-highest Should be completed ASAP

Comments

@indiv0
Copy link
Contributor

indiv0 commented Jul 1, 2024

Currently, it is possible to fetch an organization's files directly from S3 via the presigned URLs supplied by the cloud backend. These URLs are being deprecated as we are moving to SSE-C encryption for all user-supplied objects for compliance/security reasons. Presigned URLs are not easily compatible with SSE-C encryption since encryption key headers must be added to the presigned URL prior to use. We do not wish to expose SSE-C encryption keys to users, so we have added a get_file_contents lambda that proxies requests to fetch a file. This lambda adds the necessary headers and decrypts the file prior to returning it to the user.

For the relevant changes on the cloud backend:

Making requests to the new lambda:

https://github.com/enso-org/cloud-v2/blob/dc8a66281770623e1b32caa1ec3a2c0f7d3cf4da/src/lambdas/src/lambdas/files/endpoints/get_contents.rs#L37-L50

url field removed from this response body:

https://github.com/enso-org/cloud-v2/blob/dc8a66281770623e1b32caa1ec3a2c0f7d3cf4da/src/lambdas/src/lambdas/files/endpoints/get_details.rs#L66-L69

path field removed from this response body:

https://github.com/enso-org/cloud-v2/blob/dc8a66281770623e1b32caa1ec3a2c0f7d3cf4da/src/lambdas/src/lambdas/files/endpoints/upload.rs#L91-L95

It is now up to the client to construct the appropriate URL from the file_id, and requests to the new lambda must provide the JWT token for the requests to be authorized.

NOTE: To make this migration non-breaking, it is being done in two stages. First, the get_file_contents lambda with the new functionality is being added (while files are still unencrypted). This change is already deployed. Once the libs have been migrated to use the new endpoint, enso/cloud-v2#1316 will be deployed, which encrypts all the files and removes the deprecated fields from the responses.

@indiv0 indiv0 added p-highest Should be completed ASAP -libs Libraries: New libraries to be implemented labels Jul 1, 2024
@mergify mergify bot closed this as completed in #10456 Jul 9, 2024
@mergify mergify bot closed this as completed in a3dc50f Jul 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
-libs Libraries: New libraries to be implemented p-highest Should be completed ASAP
2 participants