-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(gax): add ResumableUploadClient.startUpload() and supporting types #14138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Draft
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
43 changes: 43 additions & 0 deletions
43
...m-java/gax-java/gax/src/main/java/com/google/api/gax/resumable/ResumableUploadClient.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| /* | ||
| * Copyright 2026 Google LLC | ||
| * | ||
| * Redistribution and use in source and binary forms, with or without | ||
| * modification, are permitted provided that the following conditions are | ||
| * met: | ||
| * | ||
| * * Redistributions of source code must retain the above copyright | ||
| * notice, this list of conditions and the following disclaimer. | ||
| * * Redistributions in binary form must reproduce the above | ||
| * copyright notice, this list of conditions and the following disclaimer | ||
| * in the documentation and/or other materials provided with the | ||
| * distribution. | ||
| * * Neither the name of Google LLC nor the names of its | ||
| * contributors may be used to endorse or promote products derived from | ||
| * this software without specific prior written permission. | ||
| * | ||
| * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
| * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
| * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||
| * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||
| * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||
| * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||
| * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||
| * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||
| * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
| * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| */ | ||
| package com.google.api.gax.resumable; | ||
|
|
||
| import com.google.api.core.InternalApi; | ||
| import com.google.api.gax.rpc.UnaryCallable; | ||
| import org.jspecify.annotations.NullMarked; | ||
|
|
||
| /** Client interface for executing low-level resumable upload operations. */ | ||
| @NullMarked | ||
| @InternalApi | ||
| public interface ResumableUploadClient { | ||
|
|
||
| /** Returns a {@link UnaryCallable} to initiate a resumable upload session. */ | ||
| UnaryCallable<StartUploadRequest, ResumableUploadSession> startUploadCallable(); | ||
| } |
82 changes: 82 additions & 0 deletions
82
...-java/gax-java/gax/src/main/java/com/google/api/gax/resumable/ResumableUploadSession.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| /* | ||
| * Copyright 2026 Google LLC | ||
| * | ||
| * Redistribution and use in source and binary forms, with or without | ||
| * modification, are permitted provided that the following conditions are | ||
| * met: | ||
| * | ||
| * * Redistributions of source code must retain the above copyright | ||
| * notice, this list of conditions and the following disclaimer. | ||
| * * Redistributions in binary form must reproduce the above | ||
| * copyright notice, this list of conditions and the following disclaimer | ||
| * in the documentation and/or other materials provided with the | ||
| * distribution. | ||
| * * Neither the name of Google LLC nor the names of its | ||
| * contributors may be used to endorse or promote products derived from | ||
| * this software without specific prior written permission. | ||
| * | ||
| * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
| * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
| * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||
| * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||
| * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||
| * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||
| * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||
| * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||
| * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
| * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| */ | ||
| package com.google.api.gax.resumable; | ||
|
|
||
| import com.google.api.core.InternalApi; | ||
| import com.google.auto.value.AutoValue; | ||
| import org.jspecify.annotations.NullMarked; | ||
|
|
||
| /** Represents the session metadata returned after starting a resumable upload. */ | ||
| @NullMarked | ||
| @InternalApi | ||
| @AutoValue | ||
| public abstract class ResumableUploadSession { | ||
|
|
||
| private static final long DEFAULT_CHUNK_GRANULARITY = 1L; | ||
|
|
||
| /** Returns the server-provided URL to which data uploads are directed. */ | ||
| public abstract String getUploadUrl(); | ||
|
|
||
| /** | ||
| * Returns the server-mandated chunk granularity in bytes. | ||
| * | ||
| * <p>When specified by the server (via {@code X-Goog-Upload-Chunk-Granularity}), intermediate | ||
| * upload chunks must have a size and offset that are an exact multiple of this value (the final | ||
| * chunk may be smaller). If not specified by the server, this defaults to 1 byte, indicating no | ||
| * alignment or granularity requirements apply. | ||
| * | ||
| * @return the chunk granularity in bytes | ||
| */ | ||
| public abstract long getChunkGranularity(); | ||
|
|
||
| /** | ||
| * Creates a {@link ResumableUploadSession} with the specified upload URL and default chunk | ||
| * granularity. | ||
| * | ||
| * @param uploadUrl the upload session URL | ||
| * @return a new {@link ResumableUploadSession} instance | ||
| */ | ||
| public static ResumableUploadSession create(String uploadUrl) { | ||
| return create(uploadUrl, DEFAULT_CHUNK_GRANULARITY); | ||
| } | ||
|
|
||
| /** | ||
| * Creates a {@link ResumableUploadSession} with the specified upload URL and chunk granularity. | ||
| * | ||
| * @param uploadUrl the upload session URL | ||
| * @param chunkGranularity the chunk granularity in bytes; if ≤ 0, 1 is used to indicate no | ||
| * alignment or granularity requirements apply. | ||
| * @return a new {@link ResumableUploadSession} instance | ||
| */ | ||
| public static ResumableUploadSession create(String uploadUrl, long chunkGranularity) { | ||
| return new AutoValue_ResumableUploadSession( | ||
| uploadUrl, chunkGranularity > 0 ? chunkGranularity : DEFAULT_CHUNK_GRANULARITY); | ||
| } | ||
| } |
106 changes: 106 additions & 0 deletions
106
...form-java/gax-java/gax/src/main/java/com/google/api/gax/resumable/StartUploadRequest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,106 @@ | ||||||||||||||||
| /* | ||||||||||||||||
| * Copyright 2026 Google LLC | ||||||||||||||||
| * | ||||||||||||||||
| * Redistribution and use in source and binary forms, with or without | ||||||||||||||||
| * modification, are permitted provided that the following conditions are | ||||||||||||||||
| * met: | ||||||||||||||||
| * | ||||||||||||||||
| * * Redistributions of source code must retain the above copyright | ||||||||||||||||
| * notice, this list of conditions and the following disclaimer. | ||||||||||||||||
| * * Redistributions in binary form must reproduce the above | ||||||||||||||||
| * copyright notice, this list of conditions and the following disclaimer | ||||||||||||||||
| * in the documentation and/or other materials provided with the | ||||||||||||||||
| * distribution. | ||||||||||||||||
| * * Neither the name of Google LLC nor the names of its | ||||||||||||||||
| * contributors may be used to endorse or promote products derived from | ||||||||||||||||
| * this software without specific prior written permission. | ||||||||||||||||
| * | ||||||||||||||||
| * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||||||||||||||||
| * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||||||||||||||||
| * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||||||||||||||||
| * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||||||||||||||||
| * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||||||||||||||||
| * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||||||||||||||||
| * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||||||||||||||||
| * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||||||||||||||||
| * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||||||||||||||||
| * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||||||||||||||||
| * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||||||||||||||||
| */ | ||||||||||||||||
| package com.google.api.gax.resumable; | ||||||||||||||||
|
|
||||||||||||||||
| import com.google.api.core.InternalApi; | ||||||||||||||||
| import com.google.auto.value.AutoValue; | ||||||||||||||||
| import com.google.common.collect.ImmutableList; | ||||||||||||||||
| import com.google.common.collect.ImmutableMap; | ||||||||||||||||
| import java.util.List; | ||||||||||||||||
| import java.util.Map; | ||||||||||||||||
| import org.jspecify.annotations.NullMarked; | ||||||||||||||||
| import org.jspecify.annotations.Nullable; | ||||||||||||||||
|
|
||||||||||||||||
| /** Request parameters for initiating a resumable upload session. */ | ||||||||||||||||
| @NullMarked | ||||||||||||||||
| @InternalApi | ||||||||||||||||
| @AutoValue | ||||||||||||||||
| public abstract class StartUploadRequest { | ||||||||||||||||
|
|
||||||||||||||||
| /** Returns the URL path to append to the service endpoint. */ | ||||||||||||||||
| public abstract String getPath(); | ||||||||||||||||
|
|
||||||||||||||||
| /** Returns the optional initial JSON request payload. */ | ||||||||||||||||
| @Nullable | ||||||||||||||||
| public abstract String getJsonPayload(); | ||||||||||||||||
|
|
||||||||||||||||
| /** Returns the query parameters for the initiation request. */ | ||||||||||||||||
| public abstract Map<String, List<String>> getQueryParams(); | ||||||||||||||||
|
|
||||||||||||||||
| public abstract Builder toBuilder(); | ||||||||||||||||
|
|
||||||||||||||||
| public static Builder newBuilder() { | ||||||||||||||||
| return new AutoValue_StartUploadRequest.Builder().setQueryParams(ImmutableMap.of()); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| /** | ||||||||||||||||
| * Convenience factory for creating a {@link StartUploadRequest} with only a target path. | ||||||||||||||||
| * | ||||||||||||||||
| * @param path the resource upload path | ||||||||||||||||
| * @return a new {@link StartUploadRequest} instance | ||||||||||||||||
| */ | ||||||||||||||||
| public static StartUploadRequest create(String path) { | ||||||||||||||||
| return newBuilder().setPath(path).build(); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| @AutoValue.Builder | ||||||||||||||||
| public abstract static class Builder { | ||||||||||||||||
| public abstract Builder setPath(String path); | ||||||||||||||||
|
|
||||||||||||||||
| public abstract Builder setJsonPayload(@Nullable String jsonPayload); | ||||||||||||||||
|
|
||||||||||||||||
| public abstract Builder setQueryParams(Map<String, List<String>> queryParams); | ||||||||||||||||
|
|
||||||||||||||||
| abstract @Nullable Map<String, List<String>> getQueryParams(); | ||||||||||||||||
|
|
||||||||||||||||
| abstract @Nullable String getPath(); | ||||||||||||||||
|
|
||||||||||||||||
| abstract StartUploadRequest autoBuild(); | ||||||||||||||||
|
|
||||||||||||||||
| public StartUploadRequest build() { | ||||||||||||||||
| if (getPath() != null && getPath().startsWith("/")) { | ||||||||||||||||
| setPath(getPath().substring(1)); | ||||||||||||||||
| } | ||||||||||||||||
|
Comment on lines
+88
to
+90
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. To improve readability and avoid redundant getter calls on the builder, assign the result of
Suggested change
|
||||||||||||||||
|
|
||||||||||||||||
| Map<String, List<String>> params = getQueryParams(); | ||||||||||||||||
| if (params != null && !params.isEmpty()) { | ||||||||||||||||
| ImmutableMap.Builder<String, List<String>> mapBuilder = ImmutableMap.builder(); | ||||||||||||||||
| for (Map.Entry<String, List<String>> entry : params.entrySet()) { | ||||||||||||||||
| mapBuilder.put(entry.getKey(), ImmutableList.copyOf(entry.getValue())); | ||||||||||||||||
| } | ||||||||||||||||
| setQueryParams(mapBuilder.build()); | ||||||||||||||||
| } else { | ||||||||||||||||
| setQueryParams(ImmutableMap.of()); | ||||||||||||||||
| } | ||||||||||||||||
|
|
||||||||||||||||
| return autoBuild(); | ||||||||||||||||
|
whowes marked this conversation as resolved.
|
||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
| } | ||||||||||||||||
68 changes: 68 additions & 0 deletions
68
...a/gax-java/gax/src/test/java/com/google/api/gax/resumable/ResumableUploadSessionTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| /* | ||
| * Copyright 2026 Google LLC | ||
| * | ||
| * Redistribution and use in source and binary forms, with or without | ||
| * modification, are permitted provided that the following conditions are | ||
| * met: | ||
| * | ||
| * * Redistributions of source code must retain the above copyright | ||
| * notice, this list of conditions and the following disclaimer. | ||
| * * Redistributions in binary form must reproduce the above | ||
| * copyright notice, this list of conditions and the following disclaimer | ||
| * in the documentation and/or other materials provided with the | ||
| * distribution. | ||
| * * Neither the name of Google LLC nor the names of its | ||
| * contributors may be used to endorse or promote products derived from | ||
| * this software without specific prior written permission. | ||
| * | ||
| * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | ||
| * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | ||
| * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | ||
| * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | ||
| * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | ||
| * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | ||
| * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, | ||
| * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY | ||
| * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT | ||
| * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE | ||
| * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | ||
| */ | ||
| package com.google.api.gax.resumable; | ||
|
|
||
| import static com.google.common.truth.Truth.assertThat; | ||
|
|
||
| import org.junit.jupiter.api.Test; | ||
| import org.junit.jupiter.params.ParameterizedTest; | ||
| import org.junit.jupiter.params.provider.ValueSource; | ||
|
|
||
| class ResumableUploadSessionTest { | ||
|
|
||
| private static final String UPLOAD_URL = "https://storage.googleapis.com/upload/session/12345"; | ||
|
|
||
| @Test | ||
| void create_withUploadUrl_setsDefaultChunkGranularityToOneByte() { | ||
| ResumableUploadSession session = ResumableUploadSession.create(UPLOAD_URL); | ||
|
|
||
| assertThat(session.getUploadUrl()).isEqualTo(UPLOAD_URL); | ||
| assertThat(session.getChunkGranularity()).isEqualTo(1L); | ||
| } | ||
|
|
||
| @Test | ||
| void create_withExplicitChunkGranularity_preservesGranularity() { | ||
| long customChunkGranularity = 256 * 1024L; | ||
|
|
||
| ResumableUploadSession session = | ||
| ResumableUploadSession.create(UPLOAD_URL, customChunkGranularity); | ||
|
|
||
| assertThat(session.getUploadUrl()).isEqualTo(UPLOAD_URL); | ||
| assertThat(session.getChunkGranularity()).isEqualTo(customChunkGranularity); | ||
| } | ||
|
|
||
| @ParameterizedTest | ||
| @ValueSource(longs = {0L, -1L, -100L}) | ||
| void create_withNonPositiveChunkGranularity_normalizesToOneByte(long invalidGranularity) { | ||
| ResumableUploadSession session = ResumableUploadSession.create(UPLOAD_URL, invalidGranularity); | ||
|
|
||
| assertThat(session.getChunkGranularity()).isEqualTo(1L); | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.