Deprecation Notice: The Files API feature is going to be removed at some time in the future, replaced by the Google Cloud Storage Client Library. The documentation for the deprecated API is retained for the convenience of developers still using the Files API.

This document discusses the functions that the google.appengine.api.files package provides for the Google Cloud Storage Python API.

Contents

Functions

1. files.gs.create()
2. files.listdir()
3. files.delete()
4. files.open()
5. files.File.write()
6. files.File.read()
7. files.File.tell()
8. files.File.seek()
9. files.File.close()
10. files.finalize()

Classes

1. BufferedFile()

Functions

files.gs.create (filename, mime_type='application/octet', acl=None, cache_control=None, content_encoding=None content_disposition=None, content_encoding=None, user_metadata=None)

Create a Google Cloud Storage object in the specified bucket.

Arguments

filename (Required)

The Google Cloud Storage file name for the object, in the format /gs/bucket/object_name. The object_name can contain any sequence of Unicode characters, but cannot exceed 1024 bytes when UTF-8 encoded. Avoid using control characters that are considered illegal in XML 1.0 when creating an object name, as Google Cloud Storage omits these characters when listing objects. If an object with the same name already exists, the object created by this function will overwrite the existing object.

mime_type= 'application/octet-stream' (Optional)

The MIME type of the object. If you do not specify a MIME type, the default application/octet-stream MIME type will be set for you.

acl=none (Optional)

Set a predefined ACL on your object.

If you do not set this parameter, the API sets this parameter as null and uses the default object ACL for that bucket (by default, this is project-private).

cache_control=None (Optional)

Set a Cache-Control header on your object. If you do not set this header, the Google Cloud Storage API sets:

• Cache-Control: public, max-age=3600 for publicly readable objects (objects with the ACL public-read)
• Cache-Control: no-cache, no-store, max-age=0, must-revalidate for private objects (objects with the ACL private).

content_encoding=None (Optional)

If your object is compressed, specify the compression method using the Content-Encoding header.

content_disposition=None (Optional)

Set the Content-Disposition header for your object.

user_metadata=None (Optional)

Set custom headers and values using a Dictionary data set. These headers are served on GET requests to the user in the format x-goog-meta-custom_header: custom_value. For example, you can set custom headers as follows:

params = {'author':'john', 'task':'1'}
gs.create(... user_metadata=params)

The headers are served on GET requests:

HTTP/1.1 200 OK
Cache-Control: my cache control
ETag: "7d677bb83bf43de498557f4a47570b21"
Content-Type: application/octet-stream
Last-Modified: Tue, 13 Sep 2011 17:57:04 GMT
x-goog-meta-author: john
x-goog-meta-task: 1
Content-Length: 45

Result Value

Returns a handle to the newly created object.

Back to top

files.listdir (path, prefix=None, marker=None, max_keys=None)

Returns a sorted list of the objects in the specified bucket path.

Arguments

path (Required)

The Google Cloud Storage bucket path, in the format /gs/bucketname. Supply only the bucket name in the path; do not supply subdirectories in path. You can use the prefix parameter to supply subdirectories.

prefix= None (Optional)

Specifies a pattern, resulting in the return only those file names that match the pattern. (This does not support regular expressions.) You can supply a file prefix, such as a subdirectory path.

marker= None (Optional)

The file name at which you want to start listing, with the returned list excluding the file name used as marker. One way to use this parameter is to use it with max_keys to "page through" the bucket's file names.

max_keys= None (Optional)

The maximum number of file names to be returned.

files.listdir('/gs/mybucket', prefix='/mysubdir/backups', max_keys=20)

Result Value

Returns a sorted list of file names.

Back to top

files.delete(*filenames)

Permanently deletes the specified file or files from Google Cloud Storage or Blobstore.

This works only on read-only file names.

Arguments

filenames (Required)

The file to delete, in the format of /gs/bucket/object (for Google Cloud Storage files) or /blobstore/blobkey (for Blobstore files). To delete multiple files, supply a comma-separated list of file names. (Wildcards are not supported.) For Google Cloud Storage files, you can optionally supply the output of files.listdir() directly to this function:

# delete one Google Cloud Storage file
files.delete('/gs/mybucket/mySpiffy.jpg')

# delete a list of files using listdir()
files.delete(*files.gs.listdir('/gs/mybucket', prefix='file_foo'))

Result Value

No Return value.

files.open(file, mode)

Open an object for read or write, which returns a writable or readable File object. You must open a file for read or write before you can read or write to it. If you do not specify a mode, the default mode is 'r' or read mode.

You cannot open and write to a file that has already been finalized.

Arguments

file (Required)

The file to open, in the format of /gs/bucket/object.

mode (Optional)

Pass in 'a' to open a file for write and pass in 'r' to open a file for read. For example, the following snippet opens an object and reads from it:

with files.open('/gs/mybucket/myobject/', 'r') as f:
  data = f.read(1)
  while data != "":
    print data
    data = f.read(1)
  print 'Done reading file!'

Result Value

Returns a File class that has read(), write(), and close() methods.

files.File.write(data)

Write to an object. Your object must have been created but not been finalized. You must open the object with files.open() before you can write to it.

If you are writing to an unfinalized, existent object, files.Files.write() appends to the existing data.

Arguments

data (Required)

The data to be written to the file. For example, the following snippet creates a new object and writes to it:

my_file = gs.create(file_name = '/gs/mybucket/myfile')
with files.open(my_file, 'a') as f:
  f.write('Hello World!')
  f.write('Hello World Again!')

Result Value

This function does not return any values.

Back to top

files.File.read(size)

Reads up to the specified number of bytes of an object.

Arguments

size (Required)

The number of bytes to read, as an integer. The maximum number of bytes for one API call is 32MB.

Result Value

Returns the data converted to a string. The string length is 0 when the end of the file is reached.

Back to top

files.File.tell()

Give a file's current position. Only valid when the file is opened for read.

Result Value

Returns a file's current position as an integer.

Back to top

files.File.seek(offset, whence=os.SEEK_SET)

Set a file's current position.

Arguments

offset (Required)

The position to set the file as an integer.

whence=os.SEEK_SET (Optional)

The seek mode. By default, os.SEEK_SET is used, which sets the file's position using absolute seek. Alternatively, you can use os.SEEK_CUR to set the file's position relative to the current position.

Result Value

This function does not return any values.

Back to top

files.File.close (finalize=false)

Closes a file. Closing a file does not finalize it unless you explicitly set the finalize argument. You can reopen a file that has been closed to append to it. Until you finalize a file, you cannot read from it and once you finalize a file, you can no longer append to it.

Arguments

finalize=false (Optional)

By default, your file is not finalized when you close it. If you want to close and finalize your file, you can do so by passing in a true argument.

Result Value

This function does not return any values.

Back to top

files.finalize (filename)

Closes and finalizes an object. Once an object is finalized, you cannot write to it. You must finalize an object before it can be read.

If an object has already been finalized, this call will be ignored. If the object has not been finalized, this call will close and finalize it.

Arguments

filename (Required)

The object from files.create() that has not been finalized. Note that you cannot pass an object from the files.open() method. To do so, you must call files.File.close(). For example:

path = files.gs.create('/gs/mybucket/object')
fp = files.open(path)
fp.close()
files.finalize(fp) # This does not work!
files.finalize(path) # This works!

You can also finalize an object using the handle from the files.open() method:

fp = files.open(path, 'a')
fp.close(true)

Result Value

This function does not return any values.

Back to top

Classes

BufferedFile (filename)

BufferedFile() is a file-like object that reads the underlying file in chunks. You can use to this method to read from objects as an alternative to files.File.read(), with some notable differences:

• BufferedFile() does not need to be closed
• BufferedFile() does not need to use a with statement to read the file
• BufferedFile.read(n) reads exactly n bytes unless the end of the file is reached

For example, instead of using files.File.read():

with files.open('/gs/mybucket/object') as fp:
  fp.read(1000)

You can use BufferedFile() instead:

fp = files.BufferedFile('/gs/mybucket/object')
fp.read(1000)

Arguments

filename (Required)

The file to read.

Result Value

A BufferedFile() object with seek(), tell(), and read() methods.

files.BufferedFile.read (size)

Read the underlying contents of the BufferedFile() object.

Arguments

size (Required)

The number of bytes to read as an integer. The actual number of bytes read is always equal to the size unless the end of the file is reached.

Result Value

Returns the data converted to a string. The string length is 0 when the end of the file is reached.

files.BufferedFile.seek (offset, whence=os.SEEK_SET)

This function is the same as the files.File.seek() function.

files.BufferedFile.tell()

This function is the same as the files.File.tell() function.

Back to top

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies.