Migrating to JSON API v1

Contents

1. Migration Instructions
   1. Updating HTTP Calls
   2. Updating Object Change Notifications
2. API Changes
   1. Content Encoding
   2. Downloading Objects
   3. Response Safety
   4. Redirecting
   5. Cross-Origin Resource Sharing (CORS)
   6. Cache-Control Header

Migration Instructions

For existing users of JSON API v1beta2, upgrading to v1 is a straightforward task in most cases. Before doing so, please read the sections below documenting the changes from v1beta2.

Updating HTTP Calls

If you are constructing JSON-HTTP requests directly for this API, simply replace URLs beginning "https://www.googleapis.com/storage/v1beta2" with new URLs referencing the v1 path: "https://www.googleapis.com/storage/v1".

Updating Object Change Notifications

JSON API v1beta2 includes support for object change notifications, as does v1. However, incoming notification channels are associated with particular API versions. In order to ensure continued receipt of notifications after v1beta2 has been turned down, you will need to re-issue a watch request using the v1 API for buckets that currently have v1beta2 notification channels. Because code that listens for notifications already must handle the possibility of receiving duplicate notifications (see Reliable Delivery), the following is the recommended procedure for performing this update:

1. Verify Your Compatibility

   Verify that your system can handle v1 notification messages as well as duplicate messages correctly. This should already be the case, as existing notifications may be delivered multiple times, and the only change from v1beta2 to v1 messages are the URLs in the X-Goog-Resource-Uri header, the selfLink property, and the mediaLink property. After sending a watch request to v1, you will also receive a v1 initial sync message.

2. Send a Watch Request using v1

   For each existing v1beta2 change notification channel, create an identical new notification channel with v1. You can use gsutil's notification command to accomplish this:

   gsutil -o GSUtil:json_api_version=v1 notification watchbucket {ApplicationUrl} gs://{BucketName}

   You can also subscribe programmatically using the objects#watchAll operation.

3. Verify

   Verify that you are receiving v1 notifications in addition to v1beta2 notifications. You should be receiving at least two notifications for each event, and at least one of them should reference /storage/v1 instead of /storage/v1beta2.

4. Stop v1beta2 Notification Channels

   Once you have verified that v1 notifications are being received, you should stop the notification channel that is delivering v1beta2 messages. The process is similar to sending a watch request. Here is an example using gsutil:

   gsutil -o GSUtil:json_api_version=v1beta2 notification stopchannel {channel_id} {resource_id}

   The channel_id property is passed along with each notification message via the X-Goog-Channel-Id header, and the {resource_id} header is similarly available via the X-Goog-Resource-Id property. Make sure to use the values from the v1beta2 messages and not the v1 messages.

API Changes

JSON API v1 is largely compatible with JSON API v1beta2, but there are a few differences.

• The type of "generation" fields has changed from an unsigned, 64-bit value to a signed, 64-bit value. This difference should have no impact, as generation numbers are currently smaller than 64 bits, but it may require small code changes in C++ code that uses generation fields.
• The HTTP response code for failures due to clients being rate limited has been changed from 403 Forbidden to 429 Too Many Requests.
• The behavior of uploads using the Content-Encoding header has changed. See below.

Content Encoding

When uploading objects, v1beta2 preserves the value of the Content-Encoding header in the "contentEncoding" metadata property. JSON API v1 decouples the encoding used to upload files from the encoding used to store the file and the "contentEncoding" metadata property.

When uploading objects with v1, the Content-Encoding header is only used to specify an encoding for that one upload operation and has no affect on the "contentEncoding" metadata property. Objects will be stored by v1 only after being decoded according to the rules of Content-Encoding headers.

For example, suppose you have a 10 megabyte text file that is only 1 megabyte when compressed with gzip. If you upload that file with JSON API v1beta2 with the Content-Encoding header set to "gzip" and the data appropriately compressed, the resulting object stored in Google Cloud Storage will be a 1 MB gzip archive with the "contentEncoding" metadata property set to "gzip." If you do the action with JSON API v1, the resulting object will be the original 10 megabyte text file with no "contentEncoding" metadata property.

If you want Google Cloud Storage to persist an object with its content encoding, upload it exactly as desired, but set the "contentEncoding" metadata property instead of the Content-Encoding header. For multipart or resumable uploads, set the "contentEncoding" metadata property along with the rest of the metadata. For simple uploads, use the new "contentEncoding" query parameter.

You can still use the Content-Encoding header in JSON API v1 to upload data. For example, you might want to save bandwidth by using gzip encoding to upload a large file, but you might not want to store it with that encoding in order to take advantage of features like object composition.

Downloading Objects

There have been several changes to how downloading object media is handled in JSON API v1, mostly focused on security. None of these changes affect how requests to download media are constructed. Most users should not notice a difference, but users who include calls to the JSON API from websites should read the following before updating their sites to point to v1.

Response Safety

Redirecting

When you download an object with v1beta2, your download always involves at least one redirect to another domain, including storage.googleapis.com. Under v1, this is no longer necessarily the case. Depending on a variety of conditions, mostly related to security, downloads may be served directly without a redirect or be may be redirected one or more times to other Google domains. Code that relies on being redirected exactly once or code that whitelists only the domains used with v1beta2 may have to be adjusted.

Cross-Origin Resource Sharing (CORS)

Google Cloud Storage supports customizable CORS policies for buckets which will be accessed via a domain name, either via the pattern BUCKET.storage.googleapis.com or when using CNAME aliases. Previous versions of the JSON API attempted to abide by these directives. The v1 release of the JSON API still allows users to set custom CORS policies for use with our XML endpoint, but the JSON API v1 endpoint itself does not abide by these custom policies and accepts traffic from most domains.

Cache-Control Header

The XML API, as well as v1beta2 of the JSON API, allows users to override the default Cache-Control header values with custom content. When downloading media with JSON API v1, only certain Cache-Control header values are respected, and other settings are stripped. The JSON API v1 respects the following caching directives: "public", "private", "no-cache", and "max-age". The JSON API v1 may also add an additional "no-transform" directive to downloads. Downloads via the XML API and from buckets mapped to domain names will continue to behave exactly as they did before, and the JSON API continues to allow you to set Cache-Control metadata that will be fully respected by those APIs.