• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# -*- coding: utf-8 -*-
2# Copyright 2015 Google Inc. All Rights Reserved.
3#
4# Licensed under the Apache License, Version 2.0 (the "License");
5# you may not use this file except in compliance with the License.
6# You may obtain a copy of the License at
7#
8#     http://www.apache.org/licenses/LICENSE-2.0
9#
10# Unless required by applicable law or agreed to in writing, software
11# distributed under the License is distributed on an "AS IS" BASIS,
12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13# See the License for the specific language governing permissions and
14# limitations under the License.
15"""Additional help about types of credentials and authentication."""
16
17from __future__ import absolute_import
18
19from gslib.help_provider import HelpProvider
20
21_DETAILED_HELP_TEXT = ("""
22<B>OVERVIEW</B>
23  This help section provides details about various precautions taken by gsutil
24  to protect data security, as well as recommendations for how customers should
25  safeguard security.
26
27
28<B>TRANSPORT LAYER SECURITY</B>
29  gsutil performs all operations using transport-layer encryption (HTTPS), to
30  protect against data leakage over shared network links. This is also important
31  because gsutil uses "bearer tokens" for authentication (OAuth2) as well as for
32  resumable upload identifiers, and such tokens must be protected from being
33  eavesdropped and reused.
34
35  gsutil also supports the older HMAC style of authentication via the XML API
36  (see "gsutil help apis").  While HMAC authentication does not use bearer
37  tokens (and thus is not subject to eavesdropping/replay attacks), it's still
38  important to encrypt data traffic.
39
40  Prior to gsutil release 4.0 it was possible to use HTTP instead of HTTPS by
41  setting the "is_secure" configuration parameter in the [Boto] section of the
42  boto configuration file to False. However, starting with gsutil version 4.0
43  setting is_secure to False is disallowed.  For more details about different
44  credential options, see "gsutil help creds".
45
46
47<B>LOCAL FILE STORAGE SECURITY</B>
48  gsutil takes a number of precautions to protect against security exploits in
49  the files it stores locally:
50
51    - When the gsutil config command runs it sets file protection mode 600
52      ("-rw-------") on the the .boto configuration file it generates, so only
53      the user (or superuser) can read it. This is important because these files
54      contain security-sensitive information, including credentials and proxy
55      configuration.
56
57    - The gsutil config command also uses file protection mode 600 for the
58      private key file stored locally when you create service account
59      credentials.
60
61    - The default level of logging output from gsutil commands does not include
62      security-sensitive information, such as OAuth2 tokens and proxy
63      configuration information. (See the "RECOMMENDED USER PRECAUTIONS" section
64      below if you increase the level of debug output, using the gsutil -D
65      option.)
66
67  Note that protection modes are not supported on Windows, so if you
68  use gsutil on Windows we recommend using an encrypted file system and strong
69  account passwords.
70
71
72<B>SECURITY-SENSITIVE FILES WRITTEN TEMPORARILY TO DISK BY GSUTIL</B>
73    gsutil buffers data in temporary files in several situations:
74
75      - While compressing data being uploaded via gsutil cp -z, gsutil
76        buffers the data in temporary files with protection 600, which it
77        deletes after the upload is complete (similarly for downloading files
78        that were uploaded with gsutil cp -z or some other process that sets the
79        Content-Encoding to "gzip"). However, if you kill the gsutil process
80        while the upload is under way the partially written file will be left
81        in place. See the "CHANGING TEMP DIRECTORIES" section in
82        "gsutil help cp" for details of where the temporary files are written
83        and how to change the temp directory location.
84
85      - When performing a resumable upload gsutil stores the upload ID (which,
86        as noted above, is a bearer token and thus should be safe-guarded) in a
87        file under ~/.gsutil/tracker-files with protection 600, and deletes this
88        file after the upload is complete. However, if the upload doesn't
89        complete successfully the tracker file is left in place so the resumable
90        upload can be re-attempted later. Over time it's possible to accumulate
91        these tracker files from aborted upload attempts, though resumable
92        upload IDs are only valid for 1 week, so the security risk only exists
93        for files less than that old. If you consider the risk of leaving
94        aborted upload IDs in the tracker directory too high you could modify
95        your upload scripts to delete the tracker files; or you could create a
96        cron job to clear the tracker directory periodically.
97
98      - The gsutil rsync command stores temporary files (with protection 600)
99        containing the names, sizes, and checksums of source and destination
100        directories/buckets, which it deletes after the rsync is complete.
101        However, if you kill the gsutil process while the rsync is under way the
102        listing files will be left in place.
103
104    Note that gsutil deletes temporary files using the standard OS unlink system
105    call, which does not perform `data wiping
106    <https://en.wikipedia.org/wiki/Data_erasure>`_. Thus, the content of such
107    temporary files can be recovered by a determined adversary.
108
109
110<B>ACCESS CONTROL LISTS</B>
111  Unless you specify a different ACL (e.g., via the gsutil cp -a option), by
112  default objects written to a bucket use the default object ACL on that bucket.
113  Unless you modify that ACL (e.g., via the gsutil defacl command), by default
114  it will allow all project editors write access to the object and read/write
115  access to the object's metadata; and will allow all project viewers read
116  access to the object.
117
118  The GCS access control system includes the ability to specify that objects are
119  publicly readable. Make sure you intend for any objects you write with this
120  permission to be public. Once "published", data on the Internet can be copied
121  to many places, so it's effectively impossible to regain read control over an
122  object written with this permission.
123
124  The GCS access control system includes the ability to specify that buckets are
125  publicly writable. While configuring a bucket this way can be convenient for
126  various purposes, we recommend against using this permission - it can be
127  abused for distributing illegal content, viruses, and other malware, and the
128  bucket owner is legally and financially responsible for the content stored in
129  their buckets. If you need to make content available to customers who don't
130  have Google accounts consider instead using signed URLs (see
131  "gsutil help signurl").
132
133
134<B>SOFTWARE INTEGRITY AND UPDATES</B>
135  gsutil is distributed as a standalone bundle via tar and zip files stored in
136  the gs://pub bucket, as a PyPi module, and as part of the bundled Cloud
137  SDK release. Each of these distribution methods takes a variety of security
138  precautions to protect the integrity of the software. We strongly recommend
139  against getting a copy of gsutil from any other sources (such as mirror
140  sites).
141
142
143<B>PROXY USAGE</B>
144  gsutil supports access via proxies, such as Squid and a number of commercial
145  products. A full description of their capabilities is beyond the scope of this
146  documentation, but proxies can be configured to support many security-related
147  functions, including virus scanning, Data Leakage Prevention, control over
148  which certificates/CA's are trusted, content type filtering, and many more
149  capabilities. Some of these features can slow or block legitimate gsutil
150  behavior. For example, virus scanning depends on decrypting file content,
151  which in turn requires that the proxy terminate the gsutil connection and
152  establish a new connection - and in some cases proxies will rewrite content in
153  ways that result in checksum validation errors and other problems.
154
155  For details on configuring proxies see the proxy help text in your .boto
156  configuration file (generated by the gsutil config command).
157
158
159<B>ENCRYPTION AT REST</B>
160  All GCS data are stored encrypted. For more information see
161  `Server-Side Encryption
162  <https://cloud.google.com/storage/docs/concepts-techniques#encryption>`_.
163
164
165<B>DATA PRIVACY FROM GOOGLE EMPLOYEES</B>
166  Google employees will never look at your data unless you first explicitly
167  grant them permission to do so while troubleshooting a specific incident.
168
169  Google will never ask you to share your credentials, password, or other
170  security-sensitive information. Beware of potential phishing scams where
171  someone attempts to impersonate Google and asks for such information.
172
173
174<B>MEASUREMENT DATA</B>
175  The gsutil perfdiag command collects a variety of performance-related
176  measurements and details about your local system and network environment, for
177  use in troubleshooting performance problems. None of this information will be
178  sent to Google unless you choose to send it.
179
180
181<B>RECOMMENDED USER PRECAUTIONS</B>
182  The first and foremost precaution is: Never share your credentials. Each user
183  should have distinct credentials.
184
185  If you run gsutil -D (to generate debugging output) it will include OAuth2
186  refresh and access tokens in the output. Make sure to redact this information
187  before sending this debug output to anyone during troubleshooting/tech support
188  interactions.
189
190  If you run gsutil --trace-token (to send a trace directly to Google),
191  sensitive information like OAuth2 tokens and the contents of any files
192  accessed during the trace may be included in the content of the trace.
193
194  The proxy configuration information in the .boto configuration is
195  security-sensitive, especially if your proxy setup requires user and
196  password information. Even if your proxy setup doesn't require user and
197  password, the host and port number for your proxy is often considered
198  security-sensitive. Protect access to your .boto configuration file.
199
200  If you are using gsutil from a production environment (e.g., via a cron job
201  running on a host in your data center), use service account credentials rather
202  than individual user account credentials. These credentials were designed for
203  such use and, for example, protect you from losing access when an employee
204  leaves your company.
205""")
206
207
208class CommandOptions(HelpProvider):
209  """Additional help about security and privacy considerations using gsutil."""
210
211  # Help specification. See help_provider.py for documentation.
212  help_spec = HelpProvider.HelpSpec(
213      help_name='security',
214      help_name_aliases=['encryption', 'protection', 'privacy', 'proxies',
215                         'proxy'],
216      help_type='additional_help',
217      help_one_line_summary='Security and Privacy Considerations',
218      help_text=_DETAILED_HELP_TEXT,
219      subcommand_help_text={},
220  )
221