1# -*- coding: utf-8 -*- 2# Copyright 2015 Google Inc. All Rights Reserved. 3# 4# Licensed under the Apache License, Version 2.0 (the "License"); 5# you may not use this file except in compliance with the License. 6# You may obtain a copy of the License at 7# 8# http://www.apache.org/licenses/LICENSE-2.0 9# 10# Unless required by applicable law or agreed to in writing, software 11# distributed under the License is distributed on an "AS IS" BASIS, 12# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13# See the License for the specific language governing permissions and 14# limitations under the License. 15"""Additional help about types of credentials and authentication.""" 16 17from __future__ import absolute_import 18 19from gslib.help_provider import HelpProvider 20 21_DETAILED_HELP_TEXT = (""" 22<B>OVERVIEW</B> 23 This help section provides details about various precautions taken by gsutil 24 to protect data security, as well as recommendations for how customers should 25 safeguard security. 26 27 28<B>TRANSPORT LAYER SECURITY</B> 29 gsutil performs all operations using transport-layer encryption (HTTPS), to 30 protect against data leakage over shared network links. This is also important 31 because gsutil uses "bearer tokens" for authentication (OAuth2) as well as for 32 resumable upload identifiers, and such tokens must be protected from being 33 eavesdropped and reused. 34 35 gsutil also supports the older HMAC style of authentication via the XML API 36 (see "gsutil help apis"). While HMAC authentication does not use bearer 37 tokens (and thus is not subject to eavesdropping/replay attacks), it's still 38 important to encrypt data traffic. 39 40 Prior to gsutil release 4.0 it was possible to use HTTP instead of HTTPS by 41 setting the "is_secure" configuration parameter in the [Boto] section of the 42 boto configuration file to False. However, starting with gsutil version 4.0 43 setting is_secure to False is disallowed. For more details about different 44 credential options, see "gsutil help creds". 45 46 47<B>LOCAL FILE STORAGE SECURITY</B> 48 gsutil takes a number of precautions to protect against security exploits in 49 the files it stores locally: 50 51 - When the gsutil config command runs it sets file protection mode 600 52 ("-rw-------") on the the .boto configuration file it generates, so only 53 the user (or superuser) can read it. This is important because these files 54 contain security-sensitive information, including credentials and proxy 55 configuration. 56 57 - The gsutil config command also uses file protection mode 600 for the 58 private key file stored locally when you create service account 59 credentials. 60 61 - The default level of logging output from gsutil commands does not include 62 security-sensitive information, such as OAuth2 tokens and proxy 63 configuration information. (See the "RECOMMENDED USER PRECAUTIONS" section 64 below if you increase the level of debug output, using the gsutil -D 65 option.) 66 67 Note that protection modes are not supported on Windows, so if you 68 use gsutil on Windows we recommend using an encrypted file system and strong 69 account passwords. 70 71 72<B>SECURITY-SENSITIVE FILES WRITTEN TEMPORARILY TO DISK BY GSUTIL</B> 73 gsutil buffers data in temporary files in several situations: 74 75 - While compressing data being uploaded via gsutil cp -z, gsutil 76 buffers the data in temporary files with protection 600, which it 77 deletes after the upload is complete (similarly for downloading files 78 that were uploaded with gsutil cp -z or some other process that sets the 79 Content-Encoding to "gzip"). However, if you kill the gsutil process 80 while the upload is under way the partially written file will be left 81 in place. See the "CHANGING TEMP DIRECTORIES" section in 82 "gsutil help cp" for details of where the temporary files are written 83 and how to change the temp directory location. 84 85 - When performing a resumable upload gsutil stores the upload ID (which, 86 as noted above, is a bearer token and thus should be safe-guarded) in a 87 file under ~/.gsutil/tracker-files with protection 600, and deletes this 88 file after the upload is complete. However, if the upload doesn't 89 complete successfully the tracker file is left in place so the resumable 90 upload can be re-attempted later. Over time it's possible to accumulate 91 these tracker files from aborted upload attempts, though resumable 92 upload IDs are only valid for 1 week, so the security risk only exists 93 for files less than that old. If you consider the risk of leaving 94 aborted upload IDs in the tracker directory too high you could modify 95 your upload scripts to delete the tracker files; or you could create a 96 cron job to clear the tracker directory periodically. 97 98 - The gsutil rsync command stores temporary files (with protection 600) 99 containing the names, sizes, and checksums of source and destination 100 directories/buckets, which it deletes after the rsync is complete. 101 However, if you kill the gsutil process while the rsync is under way the 102 listing files will be left in place. 103 104 Note that gsutil deletes temporary files using the standard OS unlink system 105 call, which does not perform `data wiping 106 <https://en.wikipedia.org/wiki/Data_erasure>`_. Thus, the content of such 107 temporary files can be recovered by a determined adversary. 108 109 110<B>ACCESS CONTROL LISTS</B> 111 Unless you specify a different ACL (e.g., via the gsutil cp -a option), by 112 default objects written to a bucket use the default object ACL on that bucket. 113 Unless you modify that ACL (e.g., via the gsutil defacl command), by default 114 it will allow all project editors write access to the object and read/write 115 access to the object's metadata; and will allow all project viewers read 116 access to the object. 117 118 The GCS access control system includes the ability to specify that objects are 119 publicly readable. Make sure you intend for any objects you write with this 120 permission to be public. Once "published", data on the Internet can be copied 121 to many places, so it's effectively impossible to regain read control over an 122 object written with this permission. 123 124 The GCS access control system includes the ability to specify that buckets are 125 publicly writable. While configuring a bucket this way can be convenient for 126 various purposes, we recommend against using this permission - it can be 127 abused for distributing illegal content, viruses, and other malware, and the 128 bucket owner is legally and financially responsible for the content stored in 129 their buckets. If you need to make content available to customers who don't 130 have Google accounts consider instead using signed URLs (see 131 "gsutil help signurl"). 132 133 134<B>SOFTWARE INTEGRITY AND UPDATES</B> 135 gsutil is distributed as a standalone bundle via tar and zip files stored in 136 the gs://pub bucket, as a PyPi module, and as part of the bundled Cloud 137 SDK release. Each of these distribution methods takes a variety of security 138 precautions to protect the integrity of the software. We strongly recommend 139 against getting a copy of gsutil from any other sources (such as mirror 140 sites). 141 142 143<B>PROXY USAGE</B> 144 gsutil supports access via proxies, such as Squid and a number of commercial 145 products. A full description of their capabilities is beyond the scope of this 146 documentation, but proxies can be configured to support many security-related 147 functions, including virus scanning, Data Leakage Prevention, control over 148 which certificates/CA's are trusted, content type filtering, and many more 149 capabilities. Some of these features can slow or block legitimate gsutil 150 behavior. For example, virus scanning depends on decrypting file content, 151 which in turn requires that the proxy terminate the gsutil connection and 152 establish a new connection - and in some cases proxies will rewrite content in 153 ways that result in checksum validation errors and other problems. 154 155 For details on configuring proxies see the proxy help text in your .boto 156 configuration file (generated by the gsutil config command). 157 158 159<B>ENCRYPTION AT REST</B> 160 All GCS data are stored encrypted. For more information see 161 `Server-Side Encryption 162 <https://cloud.google.com/storage/docs/concepts-techniques#encryption>`_. 163 164 165<B>DATA PRIVACY FROM GOOGLE EMPLOYEES</B> 166 Google employees will never look at your data unless you first explicitly 167 grant them permission to do so while troubleshooting a specific incident. 168 169 Google will never ask you to share your credentials, password, or other 170 security-sensitive information. Beware of potential phishing scams where 171 someone attempts to impersonate Google and asks for such information. 172 173 174<B>MEASUREMENT DATA</B> 175 The gsutil perfdiag command collects a variety of performance-related 176 measurements and details about your local system and network environment, for 177 use in troubleshooting performance problems. None of this information will be 178 sent to Google unless you choose to send it. 179 180 181<B>RECOMMENDED USER PRECAUTIONS</B> 182 The first and foremost precaution is: Never share your credentials. Each user 183 should have distinct credentials. 184 185 If you run gsutil -D (to generate debugging output) it will include OAuth2 186 refresh and access tokens in the output. Make sure to redact this information 187 before sending this debug output to anyone during troubleshooting/tech support 188 interactions. 189 190 If you run gsutil --trace-token (to send a trace directly to Google), 191 sensitive information like OAuth2 tokens and the contents of any files 192 accessed during the trace may be included in the content of the trace. 193 194 The proxy configuration information in the .boto configuration is 195 security-sensitive, especially if your proxy setup requires user and 196 password information. Even if your proxy setup doesn't require user and 197 password, the host and port number for your proxy is often considered 198 security-sensitive. Protect access to your .boto configuration file. 199 200 If you are using gsutil from a production environment (e.g., via a cron job 201 running on a host in your data center), use service account credentials rather 202 than individual user account credentials. These credentials were designed for 203 such use and, for example, protect you from losing access when an employee 204 leaves your company. 205""") 206 207 208class CommandOptions(HelpProvider): 209 """Additional help about security and privacy considerations using gsutil.""" 210 211 # Help specification. See help_provider.py for documentation. 212 help_spec = HelpProvider.HelpSpec( 213 help_name='security', 214 help_name_aliases=['encryption', 'protection', 'privacy', 'proxies', 215 'proxy'], 216 help_type='additional_help', 217 help_one_line_summary='Security and Privacy Considerations', 218 help_text=_DETAILED_HELP_TEXT, 219 subcommand_help_text={}, 220 ) 221