1<html><body> 2<style> 3 4body, h1, h2, h3, div, span, p, pre, a { 5 margin: 0; 6 padding: 0; 7 border: 0; 8 font-weight: inherit; 9 font-style: inherit; 10 font-size: 100%; 11 font-family: inherit; 12 vertical-align: baseline; 13} 14 15body { 16 font-size: 13px; 17 padding: 1em; 18} 19 20h1 { 21 font-size: 26px; 22 margin-bottom: 1em; 23} 24 25h2 { 26 font-size: 24px; 27 margin-bottom: 1em; 28} 29 30h3 { 31 font-size: 20px; 32 margin-bottom: 1em; 33 margin-top: 1em; 34} 35 36pre, code { 37 line-height: 1.5; 38 font-family: Monaco, 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Lucida Console', monospace; 39} 40 41pre { 42 margin-top: 0.5em; 43} 44 45h1, h2, h3, p { 46 font-family: Arial, sans serif; 47} 48 49h1, h2, h3 { 50 border-bottom: solid #CCC 1px; 51} 52 53.toc_element { 54 margin-top: 0.5em; 55} 56 57.firstline { 58 margin-left: 2 em; 59} 60 61.method { 62 margin-top: 1em; 63 border: solid 1px #CCC; 64 padding: 1em; 65 background: #EEE; 66} 67 68.details { 69 font-weight: bold; 70 font-size: 14px; 71} 72 73</style> 74 75<h1><a href="genomics_v1alpha2.html">Genomics API</a> . <a href="genomics_v1alpha2.pipelines.html">pipelines</a></h1> 76<h2>Instance Methods</h2> 77<p class="toc_element"> 78 <code><a href="#create">create(body, x__xgafv=None)</a></code></p> 79<p class="firstline">Creates a pipeline that can be run later. Create takes a Pipeline that</p> 80<p class="toc_element"> 81 <code><a href="#delete">delete(pipelineId, x__xgafv=None)</a></code></p> 82<p class="firstline">Deletes a pipeline based on ID.</p> 83<p class="toc_element"> 84 <code><a href="#get">get(pipelineId, x__xgafv=None)</a></code></p> 85<p class="firstline">Retrieves a pipeline based on ID.</p> 86<p class="toc_element"> 87 <code><a href="#getControllerConfig">getControllerConfig(validationToken=None, x__xgafv=None, operationId=None)</a></code></p> 88<p class="firstline">Gets controller configuration information. Should only be called</p> 89<p class="toc_element"> 90 <code><a href="#list">list(pageSize=None, projectId=None, namePrefix=None, pageToken=None, x__xgafv=None)</a></code></p> 91<p class="firstline">Lists pipelines.</p> 92<p class="toc_element"> 93 <code><a href="#list_next">list_next(previous_request, previous_response)</a></code></p> 94<p class="firstline">Retrieves the next page of results.</p> 95<p class="toc_element"> 96 <code><a href="#run">run(body, x__xgafv=None)</a></code></p> 97<p class="firstline">Runs a pipeline. If `pipelineId` is specified in the request, then</p> 98<p class="toc_element"> 99 <code><a href="#setOperationStatus">setOperationStatus(body, x__xgafv=None)</a></code></p> 100<p class="firstline">Sets status of a given operation. Any new timestamps (as determined by</p> 101<h3>Method Details</h3> 102<div class="method"> 103 <code class="details" id="create">create(body, x__xgafv=None)</code> 104 <pre>Creates a pipeline that can be run later. Create takes a Pipeline that 105has all fields other than `pipelineId` populated, and then returns 106the same pipeline with `pipelineId` populated. This id can be used 107to run the pipeline. 108 109Caller must have WRITE permission to the project. 110 111Args: 112 body: object, The request body. (required) 113 The object takes the form of: 114 115{ # The pipeline object. Represents a transformation from a set of input 116 # parameters to a set of output parameters. The transformation is defined 117 # as a docker image and command to run within that image. Each pipeline 118 # is run on a Google Compute Engine VM. A pipeline can be created with the 119 # `create` method and then later run with the `run` method, or a pipeline can 120 # be defined and run all at once with the `run` method. 121 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline 122 # is called. Cannot be specified in the Pipeline used in the 123 # CreatePipelineRequest, and will be populated in the response to 124 # CreatePipeline and all subsequent Get and List calls. Indicates that the 125 # service has registered this pipeline. 126 "description": "A String", # User-specified description. 127 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have 128 # WRITE access. 129 "outputParameters": [ # Output parameters of the pipeline. 130 { # Parameters facilitate setting and delivering data into the 131 # pipeline's execution environment. They are defined at create time, 132 # with optional defaults, and can be overridden at run time. 133 # 134 # If `localCopy` is unset, then the parameter specifies a string that 135 # is passed as-is into the pipeline, as the value of the environment 136 # variable with the given name. A default value can be optionally 137 # specified at create time. The default can be overridden at run time 138 # using the inputs map. If no default is given, a value must be 139 # supplied at runtime. 140 # 141 # If `localCopy` is defined, then the parameter specifies a data 142 # source or sink, both in Google Cloud Storage and on the Docker container 143 # where the pipeline computation is run. The service account associated with 144 # the Pipeline (by 145 # default the project's Compute Engine service account) must have access to the 146 # Google Cloud Storage paths. 147 # 148 # At run time, the Google Cloud Storage paths can be overridden if a default 149 # was provided at create time, or must be set otherwise. The pipeline runner 150 # should add a key/value pair to either the inputs or outputs map. The 151 # indicated data copies will be carried out before/after pipeline execution, 152 # just as if the corresponding arguments were provided to `gsutil cp`. 153 # 154 # For example: Given the following `PipelineParameter`, specified 155 # in the `inputParameters` list: 156 # 157 # ``` 158 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 159 # ``` 160 # 161 # where `disk` is defined in the `PipelineResources` object as: 162 # 163 # ``` 164 # {name: "pd1", mountPoint: "/mnt/disk/"} 165 # ``` 166 # 167 # We create a disk named `pd1`, mount it on the host VM, and map 168 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 169 # runtime, an entry for `input_file` would be required in the inputs 170 # map, such as: 171 # 172 # ``` 173 # inputs["input_file"] = "gs://my-bucket/bar.txt" 174 # ``` 175 # 176 # This would generate the following gsutil call: 177 # 178 # ``` 179 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 180 # ``` 181 # 182 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 183 # Docker container. Acceptable paths are: 184 # 185 # <table> 186 # <thead> 187 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 188 # </thead> 189 # <tbody> 190 # <tr><td>file</td><td>file</td></tr> 191 # <tr><td>glob</td><td>directory</td></tr> 192 # </tbody> 193 # </table> 194 # 195 # For outputs, the direction of the copy is reversed: 196 # 197 # ``` 198 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 199 # ``` 200 # 201 # Acceptable paths are: 202 # 203 # <table> 204 # <thead> 205 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 206 # </thead> 207 # <tbody> 208 # <tr><td>file</td><td>file</td></tr> 209 # <tr> 210 # <td>file</td> 211 # <td>directory - directory must already exist</td> 212 # </tr> 213 # <tr> 214 # <td>glob</td> 215 # <td>directory - directory will be created if it doesn't exist</td></tr> 216 # </tbody> 217 # </table> 218 # 219 # One restriction due to docker limitations, is that for outputs that are found 220 # on the boot disk, the local path cannot be a glob and must be a file. 221 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 222 # If `localCopy` is present, then this must be a Google Cloud Storage path 223 # beginning with `gs://`. 224 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 225 # `LocalCopy` indicates where on the VM the file should be. The value 226 # given to this parameter (either at runtime or using `defaultValue`) 227 # must be the remote path where the file should be. 228 "path": "A String", # Required. The path within the user's docker container where 229 # this input should be localized to and from, relative to the specified 230 # disk's mount point. For example: file.txt, 231 "disk": "A String", # Required. The name of the disk where this parameter is 232 # located. Can be the name of one of the disks specified in the 233 # Resources field, or "boot", which represents the Docker 234 # instance's boot disk and has a mount point of `/`. 235 }, 236 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 237 # as the key to the input and output maps in RunPipeline. 238 "description": "A String", # Human-readable description. 239 }, 240 ], 241 "inputParameters": [ # Input parameters of the pipeline. 242 { # Parameters facilitate setting and delivering data into the 243 # pipeline's execution environment. They are defined at create time, 244 # with optional defaults, and can be overridden at run time. 245 # 246 # If `localCopy` is unset, then the parameter specifies a string that 247 # is passed as-is into the pipeline, as the value of the environment 248 # variable with the given name. A default value can be optionally 249 # specified at create time. The default can be overridden at run time 250 # using the inputs map. If no default is given, a value must be 251 # supplied at runtime. 252 # 253 # If `localCopy` is defined, then the parameter specifies a data 254 # source or sink, both in Google Cloud Storage and on the Docker container 255 # where the pipeline computation is run. The service account associated with 256 # the Pipeline (by 257 # default the project's Compute Engine service account) must have access to the 258 # Google Cloud Storage paths. 259 # 260 # At run time, the Google Cloud Storage paths can be overridden if a default 261 # was provided at create time, or must be set otherwise. The pipeline runner 262 # should add a key/value pair to either the inputs or outputs map. The 263 # indicated data copies will be carried out before/after pipeline execution, 264 # just as if the corresponding arguments were provided to `gsutil cp`. 265 # 266 # For example: Given the following `PipelineParameter`, specified 267 # in the `inputParameters` list: 268 # 269 # ``` 270 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 271 # ``` 272 # 273 # where `disk` is defined in the `PipelineResources` object as: 274 # 275 # ``` 276 # {name: "pd1", mountPoint: "/mnt/disk/"} 277 # ``` 278 # 279 # We create a disk named `pd1`, mount it on the host VM, and map 280 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 281 # runtime, an entry for `input_file` would be required in the inputs 282 # map, such as: 283 # 284 # ``` 285 # inputs["input_file"] = "gs://my-bucket/bar.txt" 286 # ``` 287 # 288 # This would generate the following gsutil call: 289 # 290 # ``` 291 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 292 # ``` 293 # 294 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 295 # Docker container. Acceptable paths are: 296 # 297 # <table> 298 # <thead> 299 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 300 # </thead> 301 # <tbody> 302 # <tr><td>file</td><td>file</td></tr> 303 # <tr><td>glob</td><td>directory</td></tr> 304 # </tbody> 305 # </table> 306 # 307 # For outputs, the direction of the copy is reversed: 308 # 309 # ``` 310 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 311 # ``` 312 # 313 # Acceptable paths are: 314 # 315 # <table> 316 # <thead> 317 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 318 # </thead> 319 # <tbody> 320 # <tr><td>file</td><td>file</td></tr> 321 # <tr> 322 # <td>file</td> 323 # <td>directory - directory must already exist</td> 324 # </tr> 325 # <tr> 326 # <td>glob</td> 327 # <td>directory - directory will be created if it doesn't exist</td></tr> 328 # </tbody> 329 # </table> 330 # 331 # One restriction due to docker limitations, is that for outputs that are found 332 # on the boot disk, the local path cannot be a glob and must be a file. 333 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 334 # If `localCopy` is present, then this must be a Google Cloud Storage path 335 # beginning with `gs://`. 336 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 337 # `LocalCopy` indicates where on the VM the file should be. The value 338 # given to this parameter (either at runtime or using `defaultValue`) 339 # must be the remote path where the file should be. 340 "path": "A String", # Required. The path within the user's docker container where 341 # this input should be localized to and from, relative to the specified 342 # disk's mount point. For example: file.txt, 343 "disk": "A String", # Required. The name of the disk where this parameter is 344 # located. Can be the name of one of the disks specified in the 345 # Resources field, or "boot", which represents the Docker 346 # instance's boot disk and has a mount point of `/`. 347 }, 348 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 349 # as the key to the input and output maps in RunPipeline. 350 "description": "A String", # Human-readable description. 351 }, 352 ], 353 "docker": { # The Docker execuctor specification. # Specifies the docker run information. 354 "cmd": "A String", # Required. The command or newline delimited script to run. The command 355 # string will be executed within a bash shell. 356 # 357 # If the command exits with a non-zero exit code, output parameter 358 # de-localization will be skipped and the pipeline operation's 359 # `error` field will be populated. 360 # 361 # Maximum command string length is 16384. 362 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry. 363 # Users that run pipelines must have READ access to the image. 364 }, 365 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run. 366 # Required fields: 367 # 368 # * 369 # minimumCpuCores 370 # 371 # * 372 # minimumRamGb 373 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental 374 # feature that may go away. Defaults to false. 375 # Corresponds to `--no_address` flag for [gcloud compute instances create] 376 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). 377 # In order to use this, must be true for both create time and run time. 378 # Cannot be true at run time if false at create time. If you need to ssh into 379 # a private IP VM for debugging, you can ssh to a public VM and then ssh into 380 # the private VM's Internal IP. If noAddress is set, this pipeline run may 381 # only load docker images from Google Container Registry and not Docker Hub. 382 # Before using this, you must 383 # [configure access to Google services from internal 384 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips). 385 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB) 386 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this, 387 # must be true for both create time and run time. Cannot be true at run time 388 # if false at create time. 389 "disks": [ # Disks to attach. 390 { # A Google Compute Engine disk resource specification. 391 "name": "A String", # Required. The name of the disk that can be used in the pipeline 392 # parameters. Must be 1 - 63 characters. 393 # The name "boot" is reserved for system use. 394 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB). 395 # This field is not applicable for local SSD. 396 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end 397 # of the pipeline run, regardless of what this field is set to. 398 "source": "A String", # The full or partial URL of the persistent disk to attach. See 399 # https://cloud.google.com/compute/docs/reference/latest/instances#resource 400 # and 401 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots 402 # for more details. 403 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See 404 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances 405 # for more details. 406 # Can only be set at create time. 407 "mountPoint": "A String", # Required at create time and cannot be overridden at run time. 408 # Specifies the path in the docker container where files on 409 # this disk should be located. For example, if `mountPoint` 410 # is `/mnt/disk`, and the parameter has `localPath` 411 # `inputs/file.txt`, the docker container can access the data at 412 # `/mnt/disk/inputs/file.txt`. 413 "type": "A String", # Required. The type of the disk to create. 414 }, 415 ], 416 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB). 417 "zones": [ # List of Google Compute Engine availability zones to which resource 418 # creation will restricted. If empty, any zone may be chosen. 419 "A String", 420 ], 421 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1. 422 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type. 423 # By specifying this parameter, you will download and install the following 424 # third-party software onto your managed Compute Engine instances: NVIDIA® 425 # Tesla® drivers and NVIDIA® CUDA toolkit. 426 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of 427 # available accelerator types. 428 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach. 429 # By specifying this parameter, you will download and install the following 430 # third-party software onto your managed Compute Engine instances: 431 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit. 432 }, 433 "name": "A String", # Required. A user specified pipeline name that does not have to be unique. 434 # This name can be used for filtering Pipelines in ListPipelines. 435} 436 437 x__xgafv: string, V1 error format. 438 Allowed values 439 1 - v1 error format 440 2 - v2 error format 441 442Returns: 443 An object of the form: 444 445 { # The pipeline object. Represents a transformation from a set of input 446 # parameters to a set of output parameters. The transformation is defined 447 # as a docker image and command to run within that image. Each pipeline 448 # is run on a Google Compute Engine VM. A pipeline can be created with the 449 # `create` method and then later run with the `run` method, or a pipeline can 450 # be defined and run all at once with the `run` method. 451 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline 452 # is called. Cannot be specified in the Pipeline used in the 453 # CreatePipelineRequest, and will be populated in the response to 454 # CreatePipeline and all subsequent Get and List calls. Indicates that the 455 # service has registered this pipeline. 456 "description": "A String", # User-specified description. 457 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have 458 # WRITE access. 459 "outputParameters": [ # Output parameters of the pipeline. 460 { # Parameters facilitate setting and delivering data into the 461 # pipeline's execution environment. They are defined at create time, 462 # with optional defaults, and can be overridden at run time. 463 # 464 # If `localCopy` is unset, then the parameter specifies a string that 465 # is passed as-is into the pipeline, as the value of the environment 466 # variable with the given name. A default value can be optionally 467 # specified at create time. The default can be overridden at run time 468 # using the inputs map. If no default is given, a value must be 469 # supplied at runtime. 470 # 471 # If `localCopy` is defined, then the parameter specifies a data 472 # source or sink, both in Google Cloud Storage and on the Docker container 473 # where the pipeline computation is run. The service account associated with 474 # the Pipeline (by 475 # default the project's Compute Engine service account) must have access to the 476 # Google Cloud Storage paths. 477 # 478 # At run time, the Google Cloud Storage paths can be overridden if a default 479 # was provided at create time, or must be set otherwise. The pipeline runner 480 # should add a key/value pair to either the inputs or outputs map. The 481 # indicated data copies will be carried out before/after pipeline execution, 482 # just as if the corresponding arguments were provided to `gsutil cp`. 483 # 484 # For example: Given the following `PipelineParameter`, specified 485 # in the `inputParameters` list: 486 # 487 # ``` 488 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 489 # ``` 490 # 491 # where `disk` is defined in the `PipelineResources` object as: 492 # 493 # ``` 494 # {name: "pd1", mountPoint: "/mnt/disk/"} 495 # ``` 496 # 497 # We create a disk named `pd1`, mount it on the host VM, and map 498 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 499 # runtime, an entry for `input_file` would be required in the inputs 500 # map, such as: 501 # 502 # ``` 503 # inputs["input_file"] = "gs://my-bucket/bar.txt" 504 # ``` 505 # 506 # This would generate the following gsutil call: 507 # 508 # ``` 509 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 510 # ``` 511 # 512 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 513 # Docker container. Acceptable paths are: 514 # 515 # <table> 516 # <thead> 517 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 518 # </thead> 519 # <tbody> 520 # <tr><td>file</td><td>file</td></tr> 521 # <tr><td>glob</td><td>directory</td></tr> 522 # </tbody> 523 # </table> 524 # 525 # For outputs, the direction of the copy is reversed: 526 # 527 # ``` 528 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 529 # ``` 530 # 531 # Acceptable paths are: 532 # 533 # <table> 534 # <thead> 535 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 536 # </thead> 537 # <tbody> 538 # <tr><td>file</td><td>file</td></tr> 539 # <tr> 540 # <td>file</td> 541 # <td>directory - directory must already exist</td> 542 # </tr> 543 # <tr> 544 # <td>glob</td> 545 # <td>directory - directory will be created if it doesn't exist</td></tr> 546 # </tbody> 547 # </table> 548 # 549 # One restriction due to docker limitations, is that for outputs that are found 550 # on the boot disk, the local path cannot be a glob and must be a file. 551 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 552 # If `localCopy` is present, then this must be a Google Cloud Storage path 553 # beginning with `gs://`. 554 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 555 # `LocalCopy` indicates where on the VM the file should be. The value 556 # given to this parameter (either at runtime or using `defaultValue`) 557 # must be the remote path where the file should be. 558 "path": "A String", # Required. The path within the user's docker container where 559 # this input should be localized to and from, relative to the specified 560 # disk's mount point. For example: file.txt, 561 "disk": "A String", # Required. The name of the disk where this parameter is 562 # located. Can be the name of one of the disks specified in the 563 # Resources field, or "boot", which represents the Docker 564 # instance's boot disk and has a mount point of `/`. 565 }, 566 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 567 # as the key to the input and output maps in RunPipeline. 568 "description": "A String", # Human-readable description. 569 }, 570 ], 571 "inputParameters": [ # Input parameters of the pipeline. 572 { # Parameters facilitate setting and delivering data into the 573 # pipeline's execution environment. They are defined at create time, 574 # with optional defaults, and can be overridden at run time. 575 # 576 # If `localCopy` is unset, then the parameter specifies a string that 577 # is passed as-is into the pipeline, as the value of the environment 578 # variable with the given name. A default value can be optionally 579 # specified at create time. The default can be overridden at run time 580 # using the inputs map. If no default is given, a value must be 581 # supplied at runtime. 582 # 583 # If `localCopy` is defined, then the parameter specifies a data 584 # source or sink, both in Google Cloud Storage and on the Docker container 585 # where the pipeline computation is run. The service account associated with 586 # the Pipeline (by 587 # default the project's Compute Engine service account) must have access to the 588 # Google Cloud Storage paths. 589 # 590 # At run time, the Google Cloud Storage paths can be overridden if a default 591 # was provided at create time, or must be set otherwise. The pipeline runner 592 # should add a key/value pair to either the inputs or outputs map. The 593 # indicated data copies will be carried out before/after pipeline execution, 594 # just as if the corresponding arguments were provided to `gsutil cp`. 595 # 596 # For example: Given the following `PipelineParameter`, specified 597 # in the `inputParameters` list: 598 # 599 # ``` 600 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 601 # ``` 602 # 603 # where `disk` is defined in the `PipelineResources` object as: 604 # 605 # ``` 606 # {name: "pd1", mountPoint: "/mnt/disk/"} 607 # ``` 608 # 609 # We create a disk named `pd1`, mount it on the host VM, and map 610 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 611 # runtime, an entry for `input_file` would be required in the inputs 612 # map, such as: 613 # 614 # ``` 615 # inputs["input_file"] = "gs://my-bucket/bar.txt" 616 # ``` 617 # 618 # This would generate the following gsutil call: 619 # 620 # ``` 621 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 622 # ``` 623 # 624 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 625 # Docker container. Acceptable paths are: 626 # 627 # <table> 628 # <thead> 629 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 630 # </thead> 631 # <tbody> 632 # <tr><td>file</td><td>file</td></tr> 633 # <tr><td>glob</td><td>directory</td></tr> 634 # </tbody> 635 # </table> 636 # 637 # For outputs, the direction of the copy is reversed: 638 # 639 # ``` 640 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 641 # ``` 642 # 643 # Acceptable paths are: 644 # 645 # <table> 646 # <thead> 647 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 648 # </thead> 649 # <tbody> 650 # <tr><td>file</td><td>file</td></tr> 651 # <tr> 652 # <td>file</td> 653 # <td>directory - directory must already exist</td> 654 # </tr> 655 # <tr> 656 # <td>glob</td> 657 # <td>directory - directory will be created if it doesn't exist</td></tr> 658 # </tbody> 659 # </table> 660 # 661 # One restriction due to docker limitations, is that for outputs that are found 662 # on the boot disk, the local path cannot be a glob and must be a file. 663 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 664 # If `localCopy` is present, then this must be a Google Cloud Storage path 665 # beginning with `gs://`. 666 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 667 # `LocalCopy` indicates where on the VM the file should be. The value 668 # given to this parameter (either at runtime or using `defaultValue`) 669 # must be the remote path where the file should be. 670 "path": "A String", # Required. The path within the user's docker container where 671 # this input should be localized to and from, relative to the specified 672 # disk's mount point. For example: file.txt, 673 "disk": "A String", # Required. The name of the disk where this parameter is 674 # located. Can be the name of one of the disks specified in the 675 # Resources field, or "boot", which represents the Docker 676 # instance's boot disk and has a mount point of `/`. 677 }, 678 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 679 # as the key to the input and output maps in RunPipeline. 680 "description": "A String", # Human-readable description. 681 }, 682 ], 683 "docker": { # The Docker execuctor specification. # Specifies the docker run information. 684 "cmd": "A String", # Required. The command or newline delimited script to run. The command 685 # string will be executed within a bash shell. 686 # 687 # If the command exits with a non-zero exit code, output parameter 688 # de-localization will be skipped and the pipeline operation's 689 # `error` field will be populated. 690 # 691 # Maximum command string length is 16384. 692 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry. 693 # Users that run pipelines must have READ access to the image. 694 }, 695 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run. 696 # Required fields: 697 # 698 # * 699 # minimumCpuCores 700 # 701 # * 702 # minimumRamGb 703 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental 704 # feature that may go away. Defaults to false. 705 # Corresponds to `--no_address` flag for [gcloud compute instances create] 706 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). 707 # In order to use this, must be true for both create time and run time. 708 # Cannot be true at run time if false at create time. If you need to ssh into 709 # a private IP VM for debugging, you can ssh to a public VM and then ssh into 710 # the private VM's Internal IP. If noAddress is set, this pipeline run may 711 # only load docker images from Google Container Registry and not Docker Hub. 712 # Before using this, you must 713 # [configure access to Google services from internal 714 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips). 715 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB) 716 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this, 717 # must be true for both create time and run time. Cannot be true at run time 718 # if false at create time. 719 "disks": [ # Disks to attach. 720 { # A Google Compute Engine disk resource specification. 721 "name": "A String", # Required. The name of the disk that can be used in the pipeline 722 # parameters. Must be 1 - 63 characters. 723 # The name "boot" is reserved for system use. 724 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB). 725 # This field is not applicable for local SSD. 726 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end 727 # of the pipeline run, regardless of what this field is set to. 728 "source": "A String", # The full or partial URL of the persistent disk to attach. See 729 # https://cloud.google.com/compute/docs/reference/latest/instances#resource 730 # and 731 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots 732 # for more details. 733 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See 734 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances 735 # for more details. 736 # Can only be set at create time. 737 "mountPoint": "A String", # Required at create time and cannot be overridden at run time. 738 # Specifies the path in the docker container where files on 739 # this disk should be located. For example, if `mountPoint` 740 # is `/mnt/disk`, and the parameter has `localPath` 741 # `inputs/file.txt`, the docker container can access the data at 742 # `/mnt/disk/inputs/file.txt`. 743 "type": "A String", # Required. The type of the disk to create. 744 }, 745 ], 746 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB). 747 "zones": [ # List of Google Compute Engine availability zones to which resource 748 # creation will restricted. If empty, any zone may be chosen. 749 "A String", 750 ], 751 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1. 752 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type. 753 # By specifying this parameter, you will download and install the following 754 # third-party software onto your managed Compute Engine instances: NVIDIA® 755 # Tesla® drivers and NVIDIA® CUDA toolkit. 756 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of 757 # available accelerator types. 758 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach. 759 # By specifying this parameter, you will download and install the following 760 # third-party software onto your managed Compute Engine instances: 761 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit. 762 }, 763 "name": "A String", # Required. A user specified pipeline name that does not have to be unique. 764 # This name can be used for filtering Pipelines in ListPipelines. 765 }</pre> 766</div> 767 768<div class="method"> 769 <code class="details" id="delete">delete(pipelineId, x__xgafv=None)</code> 770 <pre>Deletes a pipeline based on ID. 771 772Caller must have WRITE permission to the project. 773 774Args: 775 pipelineId: string, Caller must have WRITE access to the project in which this pipeline 776is defined. (required) 777 x__xgafv: string, V1 error format. 778 Allowed values 779 1 - v1 error format 780 2 - v2 error format 781 782Returns: 783 An object of the form: 784 785 { # A generic empty message that you can re-use to avoid defining duplicated 786 # empty messages in your APIs. A typical example is to use it as the request 787 # or the response type of an API method. For instance: 788 # 789 # service Foo { 790 # rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); 791 # } 792 # 793 # The JSON representation for `Empty` is empty JSON object `{}`. 794 }</pre> 795</div> 796 797<div class="method"> 798 <code class="details" id="get">get(pipelineId, x__xgafv=None)</code> 799 <pre>Retrieves a pipeline based on ID. 800 801Caller must have READ permission to the project. 802 803Args: 804 pipelineId: string, Caller must have READ access to the project in which this pipeline 805is defined. (required) 806 x__xgafv: string, V1 error format. 807 Allowed values 808 1 - v1 error format 809 2 - v2 error format 810 811Returns: 812 An object of the form: 813 814 { # The pipeline object. Represents a transformation from a set of input 815 # parameters to a set of output parameters. The transformation is defined 816 # as a docker image and command to run within that image. Each pipeline 817 # is run on a Google Compute Engine VM. A pipeline can be created with the 818 # `create` method and then later run with the `run` method, or a pipeline can 819 # be defined and run all at once with the `run` method. 820 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline 821 # is called. Cannot be specified in the Pipeline used in the 822 # CreatePipelineRequest, and will be populated in the response to 823 # CreatePipeline and all subsequent Get and List calls. Indicates that the 824 # service has registered this pipeline. 825 "description": "A String", # User-specified description. 826 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have 827 # WRITE access. 828 "outputParameters": [ # Output parameters of the pipeline. 829 { # Parameters facilitate setting and delivering data into the 830 # pipeline's execution environment. They are defined at create time, 831 # with optional defaults, and can be overridden at run time. 832 # 833 # If `localCopy` is unset, then the parameter specifies a string that 834 # is passed as-is into the pipeline, as the value of the environment 835 # variable with the given name. A default value can be optionally 836 # specified at create time. The default can be overridden at run time 837 # using the inputs map. If no default is given, a value must be 838 # supplied at runtime. 839 # 840 # If `localCopy` is defined, then the parameter specifies a data 841 # source or sink, both in Google Cloud Storage and on the Docker container 842 # where the pipeline computation is run. The service account associated with 843 # the Pipeline (by 844 # default the project's Compute Engine service account) must have access to the 845 # Google Cloud Storage paths. 846 # 847 # At run time, the Google Cloud Storage paths can be overridden if a default 848 # was provided at create time, or must be set otherwise. The pipeline runner 849 # should add a key/value pair to either the inputs or outputs map. The 850 # indicated data copies will be carried out before/after pipeline execution, 851 # just as if the corresponding arguments were provided to `gsutil cp`. 852 # 853 # For example: Given the following `PipelineParameter`, specified 854 # in the `inputParameters` list: 855 # 856 # ``` 857 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 858 # ``` 859 # 860 # where `disk` is defined in the `PipelineResources` object as: 861 # 862 # ``` 863 # {name: "pd1", mountPoint: "/mnt/disk/"} 864 # ``` 865 # 866 # We create a disk named `pd1`, mount it on the host VM, and map 867 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 868 # runtime, an entry for `input_file` would be required in the inputs 869 # map, such as: 870 # 871 # ``` 872 # inputs["input_file"] = "gs://my-bucket/bar.txt" 873 # ``` 874 # 875 # This would generate the following gsutil call: 876 # 877 # ``` 878 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 879 # ``` 880 # 881 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 882 # Docker container. Acceptable paths are: 883 # 884 # <table> 885 # <thead> 886 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 887 # </thead> 888 # <tbody> 889 # <tr><td>file</td><td>file</td></tr> 890 # <tr><td>glob</td><td>directory</td></tr> 891 # </tbody> 892 # </table> 893 # 894 # For outputs, the direction of the copy is reversed: 895 # 896 # ``` 897 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 898 # ``` 899 # 900 # Acceptable paths are: 901 # 902 # <table> 903 # <thead> 904 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 905 # </thead> 906 # <tbody> 907 # <tr><td>file</td><td>file</td></tr> 908 # <tr> 909 # <td>file</td> 910 # <td>directory - directory must already exist</td> 911 # </tr> 912 # <tr> 913 # <td>glob</td> 914 # <td>directory - directory will be created if it doesn't exist</td></tr> 915 # </tbody> 916 # </table> 917 # 918 # One restriction due to docker limitations, is that for outputs that are found 919 # on the boot disk, the local path cannot be a glob and must be a file. 920 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 921 # If `localCopy` is present, then this must be a Google Cloud Storage path 922 # beginning with `gs://`. 923 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 924 # `LocalCopy` indicates where on the VM the file should be. The value 925 # given to this parameter (either at runtime or using `defaultValue`) 926 # must be the remote path where the file should be. 927 "path": "A String", # Required. The path within the user's docker container where 928 # this input should be localized to and from, relative to the specified 929 # disk's mount point. For example: file.txt, 930 "disk": "A String", # Required. The name of the disk where this parameter is 931 # located. Can be the name of one of the disks specified in the 932 # Resources field, or "boot", which represents the Docker 933 # instance's boot disk and has a mount point of `/`. 934 }, 935 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 936 # as the key to the input and output maps in RunPipeline. 937 "description": "A String", # Human-readable description. 938 }, 939 ], 940 "inputParameters": [ # Input parameters of the pipeline. 941 { # Parameters facilitate setting and delivering data into the 942 # pipeline's execution environment. They are defined at create time, 943 # with optional defaults, and can be overridden at run time. 944 # 945 # If `localCopy` is unset, then the parameter specifies a string that 946 # is passed as-is into the pipeline, as the value of the environment 947 # variable with the given name. A default value can be optionally 948 # specified at create time. The default can be overridden at run time 949 # using the inputs map. If no default is given, a value must be 950 # supplied at runtime. 951 # 952 # If `localCopy` is defined, then the parameter specifies a data 953 # source or sink, both in Google Cloud Storage and on the Docker container 954 # where the pipeline computation is run. The service account associated with 955 # the Pipeline (by 956 # default the project's Compute Engine service account) must have access to the 957 # Google Cloud Storage paths. 958 # 959 # At run time, the Google Cloud Storage paths can be overridden if a default 960 # was provided at create time, or must be set otherwise. The pipeline runner 961 # should add a key/value pair to either the inputs or outputs map. The 962 # indicated data copies will be carried out before/after pipeline execution, 963 # just as if the corresponding arguments were provided to `gsutil cp`. 964 # 965 # For example: Given the following `PipelineParameter`, specified 966 # in the `inputParameters` list: 967 # 968 # ``` 969 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 970 # ``` 971 # 972 # where `disk` is defined in the `PipelineResources` object as: 973 # 974 # ``` 975 # {name: "pd1", mountPoint: "/mnt/disk/"} 976 # ``` 977 # 978 # We create a disk named `pd1`, mount it on the host VM, and map 979 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 980 # runtime, an entry for `input_file` would be required in the inputs 981 # map, such as: 982 # 983 # ``` 984 # inputs["input_file"] = "gs://my-bucket/bar.txt" 985 # ``` 986 # 987 # This would generate the following gsutil call: 988 # 989 # ``` 990 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 991 # ``` 992 # 993 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 994 # Docker container. Acceptable paths are: 995 # 996 # <table> 997 # <thead> 998 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 999 # </thead> 1000 # <tbody> 1001 # <tr><td>file</td><td>file</td></tr> 1002 # <tr><td>glob</td><td>directory</td></tr> 1003 # </tbody> 1004 # </table> 1005 # 1006 # For outputs, the direction of the copy is reversed: 1007 # 1008 # ``` 1009 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 1010 # ``` 1011 # 1012 # Acceptable paths are: 1013 # 1014 # <table> 1015 # <thead> 1016 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 1017 # </thead> 1018 # <tbody> 1019 # <tr><td>file</td><td>file</td></tr> 1020 # <tr> 1021 # <td>file</td> 1022 # <td>directory - directory must already exist</td> 1023 # </tr> 1024 # <tr> 1025 # <td>glob</td> 1026 # <td>directory - directory will be created if it doesn't exist</td></tr> 1027 # </tbody> 1028 # </table> 1029 # 1030 # One restriction due to docker limitations, is that for outputs that are found 1031 # on the boot disk, the local path cannot be a glob and must be a file. 1032 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 1033 # If `localCopy` is present, then this must be a Google Cloud Storage path 1034 # beginning with `gs://`. 1035 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 1036 # `LocalCopy` indicates where on the VM the file should be. The value 1037 # given to this parameter (either at runtime or using `defaultValue`) 1038 # must be the remote path where the file should be. 1039 "path": "A String", # Required. The path within the user's docker container where 1040 # this input should be localized to and from, relative to the specified 1041 # disk's mount point. For example: file.txt, 1042 "disk": "A String", # Required. The name of the disk where this parameter is 1043 # located. Can be the name of one of the disks specified in the 1044 # Resources field, or "boot", which represents the Docker 1045 # instance's boot disk and has a mount point of `/`. 1046 }, 1047 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 1048 # as the key to the input and output maps in RunPipeline. 1049 "description": "A String", # Human-readable description. 1050 }, 1051 ], 1052 "docker": { # The Docker execuctor specification. # Specifies the docker run information. 1053 "cmd": "A String", # Required. The command or newline delimited script to run. The command 1054 # string will be executed within a bash shell. 1055 # 1056 # If the command exits with a non-zero exit code, output parameter 1057 # de-localization will be skipped and the pipeline operation's 1058 # `error` field will be populated. 1059 # 1060 # Maximum command string length is 16384. 1061 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry. 1062 # Users that run pipelines must have READ access to the image. 1063 }, 1064 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run. 1065 # Required fields: 1066 # 1067 # * 1068 # minimumCpuCores 1069 # 1070 # * 1071 # minimumRamGb 1072 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental 1073 # feature that may go away. Defaults to false. 1074 # Corresponds to `--no_address` flag for [gcloud compute instances create] 1075 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). 1076 # In order to use this, must be true for both create time and run time. 1077 # Cannot be true at run time if false at create time. If you need to ssh into 1078 # a private IP VM for debugging, you can ssh to a public VM and then ssh into 1079 # the private VM's Internal IP. If noAddress is set, this pipeline run may 1080 # only load docker images from Google Container Registry and not Docker Hub. 1081 # Before using this, you must 1082 # [configure access to Google services from internal 1083 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips). 1084 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB) 1085 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this, 1086 # must be true for both create time and run time. Cannot be true at run time 1087 # if false at create time. 1088 "disks": [ # Disks to attach. 1089 { # A Google Compute Engine disk resource specification. 1090 "name": "A String", # Required. The name of the disk that can be used in the pipeline 1091 # parameters. Must be 1 - 63 characters. 1092 # The name "boot" is reserved for system use. 1093 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB). 1094 # This field is not applicable for local SSD. 1095 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end 1096 # of the pipeline run, regardless of what this field is set to. 1097 "source": "A String", # The full or partial URL of the persistent disk to attach. See 1098 # https://cloud.google.com/compute/docs/reference/latest/instances#resource 1099 # and 1100 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots 1101 # for more details. 1102 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See 1103 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances 1104 # for more details. 1105 # Can only be set at create time. 1106 "mountPoint": "A String", # Required at create time and cannot be overridden at run time. 1107 # Specifies the path in the docker container where files on 1108 # this disk should be located. For example, if `mountPoint` 1109 # is `/mnt/disk`, and the parameter has `localPath` 1110 # `inputs/file.txt`, the docker container can access the data at 1111 # `/mnt/disk/inputs/file.txt`. 1112 "type": "A String", # Required. The type of the disk to create. 1113 }, 1114 ], 1115 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB). 1116 "zones": [ # List of Google Compute Engine availability zones to which resource 1117 # creation will restricted. If empty, any zone may be chosen. 1118 "A String", 1119 ], 1120 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1. 1121 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type. 1122 # By specifying this parameter, you will download and install the following 1123 # third-party software onto your managed Compute Engine instances: NVIDIA® 1124 # Tesla® drivers and NVIDIA® CUDA toolkit. 1125 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of 1126 # available accelerator types. 1127 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach. 1128 # By specifying this parameter, you will download and install the following 1129 # third-party software onto your managed Compute Engine instances: 1130 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit. 1131 }, 1132 "name": "A String", # Required. A user specified pipeline name that does not have to be unique. 1133 # This name can be used for filtering Pipelines in ListPipelines. 1134 }</pre> 1135</div> 1136 1137<div class="method"> 1138 <code class="details" id="getControllerConfig">getControllerConfig(validationToken=None, x__xgafv=None, operationId=None)</code> 1139 <pre>Gets controller configuration information. Should only be called 1140by VMs created by the Pipelines Service and not by end users. 1141 1142Args: 1143 validationToken: string, A parameter 1144 x__xgafv: string, V1 error format. 1145 Allowed values 1146 1 - v1 error format 1147 2 - v2 error format 1148 operationId: string, The operation to retrieve controller configuration for. 1149 1150Returns: 1151 An object of the form: 1152 1153 { # Stores the information that the controller will fetch from the 1154 # server in order to run. Should only be used by VMs created by the 1155 # Pipelines Service and not by end users. 1156 "cmd": "A String", 1157 "machineType": "A String", 1158 "vars": { 1159 "a_key": "A String", 1160 }, 1161 "gcsLogPath": "A String", 1162 "gcsSinks": { 1163 "a_key": { 1164 "values": [ 1165 "A String", 1166 ], 1167 }, 1168 }, 1169 "image": "A String", 1170 "disks": { 1171 "a_key": "A String", 1172 }, 1173 "gcsSources": { 1174 "a_key": { 1175 "values": [ 1176 "A String", 1177 ], 1178 }, 1179 }, 1180 }</pre> 1181</div> 1182 1183<div class="method"> 1184 <code class="details" id="list">list(pageSize=None, projectId=None, namePrefix=None, pageToken=None, x__xgafv=None)</code> 1185 <pre>Lists pipelines. 1186 1187Caller must have READ permission to the project. 1188 1189Args: 1190 pageSize: integer, Number of pipelines to return at once. Defaults to 256, and max 1191is 2048. 1192 projectId: string, Required. The name of the project to search for pipelines. Caller 1193must have READ access to this project. 1194 namePrefix: string, Pipelines with names that match this prefix should be 1195returned. If unspecified, all pipelines in the project, up to 1196`pageSize`, will be returned. 1197 pageToken: string, Token to use to indicate where to start getting results. 1198If unspecified, returns the first page of results. 1199 x__xgafv: string, V1 error format. 1200 Allowed values 1201 1 - v1 error format 1202 2 - v2 error format 1203 1204Returns: 1205 An object of the form: 1206 1207 { # The response of ListPipelines. Contains at most `pageSize` 1208 # pipelines. If it contains `pageSize` pipelines, and more pipelines 1209 # exist, then `nextPageToken` will be populated and should be 1210 # used as the `pageToken` argument to a subsequent ListPipelines 1211 # request. 1212 "nextPageToken": "A String", # The token to use to get the next page of results. 1213 "pipelines": [ # The matched pipelines. 1214 { # The pipeline object. Represents a transformation from a set of input 1215 # parameters to a set of output parameters. The transformation is defined 1216 # as a docker image and command to run within that image. Each pipeline 1217 # is run on a Google Compute Engine VM. A pipeline can be created with the 1218 # `create` method and then later run with the `run` method, or a pipeline can 1219 # be defined and run all at once with the `run` method. 1220 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline 1221 # is called. Cannot be specified in the Pipeline used in the 1222 # CreatePipelineRequest, and will be populated in the response to 1223 # CreatePipeline and all subsequent Get and List calls. Indicates that the 1224 # service has registered this pipeline. 1225 "description": "A String", # User-specified description. 1226 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have 1227 # WRITE access. 1228 "outputParameters": [ # Output parameters of the pipeline. 1229 { # Parameters facilitate setting and delivering data into the 1230 # pipeline's execution environment. They are defined at create time, 1231 # with optional defaults, and can be overridden at run time. 1232 # 1233 # If `localCopy` is unset, then the parameter specifies a string that 1234 # is passed as-is into the pipeline, as the value of the environment 1235 # variable with the given name. A default value can be optionally 1236 # specified at create time. The default can be overridden at run time 1237 # using the inputs map. If no default is given, a value must be 1238 # supplied at runtime. 1239 # 1240 # If `localCopy` is defined, then the parameter specifies a data 1241 # source or sink, both in Google Cloud Storage and on the Docker container 1242 # where the pipeline computation is run. The service account associated with 1243 # the Pipeline (by 1244 # default the project's Compute Engine service account) must have access to the 1245 # Google Cloud Storage paths. 1246 # 1247 # At run time, the Google Cloud Storage paths can be overridden if a default 1248 # was provided at create time, or must be set otherwise. The pipeline runner 1249 # should add a key/value pair to either the inputs or outputs map. The 1250 # indicated data copies will be carried out before/after pipeline execution, 1251 # just as if the corresponding arguments were provided to `gsutil cp`. 1252 # 1253 # For example: Given the following `PipelineParameter`, specified 1254 # in the `inputParameters` list: 1255 # 1256 # ``` 1257 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 1258 # ``` 1259 # 1260 # where `disk` is defined in the `PipelineResources` object as: 1261 # 1262 # ``` 1263 # {name: "pd1", mountPoint: "/mnt/disk/"} 1264 # ``` 1265 # 1266 # We create a disk named `pd1`, mount it on the host VM, and map 1267 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 1268 # runtime, an entry for `input_file` would be required in the inputs 1269 # map, such as: 1270 # 1271 # ``` 1272 # inputs["input_file"] = "gs://my-bucket/bar.txt" 1273 # ``` 1274 # 1275 # This would generate the following gsutil call: 1276 # 1277 # ``` 1278 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 1279 # ``` 1280 # 1281 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 1282 # Docker container. Acceptable paths are: 1283 # 1284 # <table> 1285 # <thead> 1286 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 1287 # </thead> 1288 # <tbody> 1289 # <tr><td>file</td><td>file</td></tr> 1290 # <tr><td>glob</td><td>directory</td></tr> 1291 # </tbody> 1292 # </table> 1293 # 1294 # For outputs, the direction of the copy is reversed: 1295 # 1296 # ``` 1297 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 1298 # ``` 1299 # 1300 # Acceptable paths are: 1301 # 1302 # <table> 1303 # <thead> 1304 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 1305 # </thead> 1306 # <tbody> 1307 # <tr><td>file</td><td>file</td></tr> 1308 # <tr> 1309 # <td>file</td> 1310 # <td>directory - directory must already exist</td> 1311 # </tr> 1312 # <tr> 1313 # <td>glob</td> 1314 # <td>directory - directory will be created if it doesn't exist</td></tr> 1315 # </tbody> 1316 # </table> 1317 # 1318 # One restriction due to docker limitations, is that for outputs that are found 1319 # on the boot disk, the local path cannot be a glob and must be a file. 1320 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 1321 # If `localCopy` is present, then this must be a Google Cloud Storage path 1322 # beginning with `gs://`. 1323 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 1324 # `LocalCopy` indicates where on the VM the file should be. The value 1325 # given to this parameter (either at runtime or using `defaultValue`) 1326 # must be the remote path where the file should be. 1327 "path": "A String", # Required. The path within the user's docker container where 1328 # this input should be localized to and from, relative to the specified 1329 # disk's mount point. For example: file.txt, 1330 "disk": "A String", # Required. The name of the disk where this parameter is 1331 # located. Can be the name of one of the disks specified in the 1332 # Resources field, or "boot", which represents the Docker 1333 # instance's boot disk and has a mount point of `/`. 1334 }, 1335 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 1336 # as the key to the input and output maps in RunPipeline. 1337 "description": "A String", # Human-readable description. 1338 }, 1339 ], 1340 "inputParameters": [ # Input parameters of the pipeline. 1341 { # Parameters facilitate setting and delivering data into the 1342 # pipeline's execution environment. They are defined at create time, 1343 # with optional defaults, and can be overridden at run time. 1344 # 1345 # If `localCopy` is unset, then the parameter specifies a string that 1346 # is passed as-is into the pipeline, as the value of the environment 1347 # variable with the given name. A default value can be optionally 1348 # specified at create time. The default can be overridden at run time 1349 # using the inputs map. If no default is given, a value must be 1350 # supplied at runtime. 1351 # 1352 # If `localCopy` is defined, then the parameter specifies a data 1353 # source or sink, both in Google Cloud Storage and on the Docker container 1354 # where the pipeline computation is run. The service account associated with 1355 # the Pipeline (by 1356 # default the project's Compute Engine service account) must have access to the 1357 # Google Cloud Storage paths. 1358 # 1359 # At run time, the Google Cloud Storage paths can be overridden if a default 1360 # was provided at create time, or must be set otherwise. The pipeline runner 1361 # should add a key/value pair to either the inputs or outputs map. The 1362 # indicated data copies will be carried out before/after pipeline execution, 1363 # just as if the corresponding arguments were provided to `gsutil cp`. 1364 # 1365 # For example: Given the following `PipelineParameter`, specified 1366 # in the `inputParameters` list: 1367 # 1368 # ``` 1369 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 1370 # ``` 1371 # 1372 # where `disk` is defined in the `PipelineResources` object as: 1373 # 1374 # ``` 1375 # {name: "pd1", mountPoint: "/mnt/disk/"} 1376 # ``` 1377 # 1378 # We create a disk named `pd1`, mount it on the host VM, and map 1379 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 1380 # runtime, an entry for `input_file` would be required in the inputs 1381 # map, such as: 1382 # 1383 # ``` 1384 # inputs["input_file"] = "gs://my-bucket/bar.txt" 1385 # ``` 1386 # 1387 # This would generate the following gsutil call: 1388 # 1389 # ``` 1390 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 1391 # ``` 1392 # 1393 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 1394 # Docker container. Acceptable paths are: 1395 # 1396 # <table> 1397 # <thead> 1398 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 1399 # </thead> 1400 # <tbody> 1401 # <tr><td>file</td><td>file</td></tr> 1402 # <tr><td>glob</td><td>directory</td></tr> 1403 # </tbody> 1404 # </table> 1405 # 1406 # For outputs, the direction of the copy is reversed: 1407 # 1408 # ``` 1409 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 1410 # ``` 1411 # 1412 # Acceptable paths are: 1413 # 1414 # <table> 1415 # <thead> 1416 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 1417 # </thead> 1418 # <tbody> 1419 # <tr><td>file</td><td>file</td></tr> 1420 # <tr> 1421 # <td>file</td> 1422 # <td>directory - directory must already exist</td> 1423 # </tr> 1424 # <tr> 1425 # <td>glob</td> 1426 # <td>directory - directory will be created if it doesn't exist</td></tr> 1427 # </tbody> 1428 # </table> 1429 # 1430 # One restriction due to docker limitations, is that for outputs that are found 1431 # on the boot disk, the local path cannot be a glob and must be a file. 1432 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 1433 # If `localCopy` is present, then this must be a Google Cloud Storage path 1434 # beginning with `gs://`. 1435 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 1436 # `LocalCopy` indicates where on the VM the file should be. The value 1437 # given to this parameter (either at runtime or using `defaultValue`) 1438 # must be the remote path where the file should be. 1439 "path": "A String", # Required. The path within the user's docker container where 1440 # this input should be localized to and from, relative to the specified 1441 # disk's mount point. For example: file.txt, 1442 "disk": "A String", # Required. The name of the disk where this parameter is 1443 # located. Can be the name of one of the disks specified in the 1444 # Resources field, or "boot", which represents the Docker 1445 # instance's boot disk and has a mount point of `/`. 1446 }, 1447 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 1448 # as the key to the input and output maps in RunPipeline. 1449 "description": "A String", # Human-readable description. 1450 }, 1451 ], 1452 "docker": { # The Docker execuctor specification. # Specifies the docker run information. 1453 "cmd": "A String", # Required. The command or newline delimited script to run. The command 1454 # string will be executed within a bash shell. 1455 # 1456 # If the command exits with a non-zero exit code, output parameter 1457 # de-localization will be skipped and the pipeline operation's 1458 # `error` field will be populated. 1459 # 1460 # Maximum command string length is 16384. 1461 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry. 1462 # Users that run pipelines must have READ access to the image. 1463 }, 1464 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run. 1465 # Required fields: 1466 # 1467 # * 1468 # minimumCpuCores 1469 # 1470 # * 1471 # minimumRamGb 1472 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental 1473 # feature that may go away. Defaults to false. 1474 # Corresponds to `--no_address` flag for [gcloud compute instances create] 1475 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). 1476 # In order to use this, must be true for both create time and run time. 1477 # Cannot be true at run time if false at create time. If you need to ssh into 1478 # a private IP VM for debugging, you can ssh to a public VM and then ssh into 1479 # the private VM's Internal IP. If noAddress is set, this pipeline run may 1480 # only load docker images from Google Container Registry and not Docker Hub. 1481 # Before using this, you must 1482 # [configure access to Google services from internal 1483 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips). 1484 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB) 1485 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this, 1486 # must be true for both create time and run time. Cannot be true at run time 1487 # if false at create time. 1488 "disks": [ # Disks to attach. 1489 { # A Google Compute Engine disk resource specification. 1490 "name": "A String", # Required. The name of the disk that can be used in the pipeline 1491 # parameters. Must be 1 - 63 characters. 1492 # The name "boot" is reserved for system use. 1493 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB). 1494 # This field is not applicable for local SSD. 1495 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end 1496 # of the pipeline run, regardless of what this field is set to. 1497 "source": "A String", # The full or partial URL of the persistent disk to attach. See 1498 # https://cloud.google.com/compute/docs/reference/latest/instances#resource 1499 # and 1500 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots 1501 # for more details. 1502 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See 1503 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances 1504 # for more details. 1505 # Can only be set at create time. 1506 "mountPoint": "A String", # Required at create time and cannot be overridden at run time. 1507 # Specifies the path in the docker container where files on 1508 # this disk should be located. For example, if `mountPoint` 1509 # is `/mnt/disk`, and the parameter has `localPath` 1510 # `inputs/file.txt`, the docker container can access the data at 1511 # `/mnt/disk/inputs/file.txt`. 1512 "type": "A String", # Required. The type of the disk to create. 1513 }, 1514 ], 1515 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB). 1516 "zones": [ # List of Google Compute Engine availability zones to which resource 1517 # creation will restricted. If empty, any zone may be chosen. 1518 "A String", 1519 ], 1520 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1. 1521 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type. 1522 # By specifying this parameter, you will download and install the following 1523 # third-party software onto your managed Compute Engine instances: NVIDIA® 1524 # Tesla® drivers and NVIDIA® CUDA toolkit. 1525 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of 1526 # available accelerator types. 1527 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach. 1528 # By specifying this parameter, you will download and install the following 1529 # third-party software onto your managed Compute Engine instances: 1530 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit. 1531 }, 1532 "name": "A String", # Required. A user specified pipeline name that does not have to be unique. 1533 # This name can be used for filtering Pipelines in ListPipelines. 1534 }, 1535 ], 1536 }</pre> 1537</div> 1538 1539<div class="method"> 1540 <code class="details" id="list_next">list_next(previous_request, previous_response)</code> 1541 <pre>Retrieves the next page of results. 1542 1543Args: 1544 previous_request: The request for the previous page. (required) 1545 previous_response: The response from the request for the previous page. (required) 1546 1547Returns: 1548 A request object that you can call 'execute()' on to request the next 1549 page. Returns None if there are no more items in the collection. 1550 </pre> 1551</div> 1552 1553<div class="method"> 1554 <code class="details" id="run">run(body, x__xgafv=None)</code> 1555 <pre>Runs a pipeline. If `pipelineId` is specified in the request, then 1556run a saved pipeline. If `ephemeralPipeline` is specified, then run 1557that pipeline once without saving a copy. 1558 1559The caller must have READ permission to the project where the pipeline 1560is stored and WRITE permission to the project where the pipeline will be 1561run, as VMs will be created and storage will be used. 1562 1563If a pipeline operation is still running after 6 days, it will be canceled. 1564 1565Args: 1566 body: object, The request body. (required) 1567 The object takes the form of: 1568 1569{ # The request to run a pipeline. If `pipelineId` is specified, it 1570 # refers to a saved pipeline created with CreatePipeline and set as 1571 # the `pipelineId` of the returned Pipeline object. If 1572 # `ephemeralPipeline` is specified, that pipeline is run once 1573 # with the given args and not saved. It is an error to specify both 1574 # `pipelineId` and `ephemeralPipeline`. `pipelineArgs` 1575 # must be specified. 1576 "pipelineArgs": { # The pipeline run arguments. # The arguments to use when running this pipeline. 1577 "inputs": { # Pipeline input arguments; keys are defined in the pipeline documentation. 1578 # All input parameters that do not have default values must be specified. 1579 # If parameters with defaults are specified here, the defaults will be 1580 # overridden. 1581 "a_key": "A String", 1582 }, 1583 "logging": { # The logging options for the pipeline run. # Required. Logging options. Used by the service to communicate results 1584 # to the user. 1585 "gcsPath": "A String", # The location in Google Cloud Storage to which the pipeline logs 1586 # will be copied. Can be specified as a fully qualified directory 1587 # path, in which case logs will be output with a unique identifier 1588 # as the filename in that directory, or as a fully specified path, 1589 # which must end in `.log`, in which case that path will be 1590 # used, and the user must ensure that logs are not 1591 # overwritten. Stdout and stderr logs from the run are also 1592 # generated and output as `-stdout.log` and `-stderr.log`. 1593 }, 1594 "outputs": { # Pipeline output arguments; keys are defined in the pipeline 1595 # documentation. All output parameters of without default values 1596 # must be specified. If parameters with defaults are specified 1597 # here, the defaults will be overridden. 1598 "a_key": "A String", 1599 }, 1600 "labels": { # Labels to apply to this pipeline run. Labels will also be applied to 1601 # compute resources (VM, disks) created by this pipeline run. When listing 1602 # operations, operations can filtered by labels. 1603 # Label keys may not be empty; label values may be empty. Non-empty labels 1604 # must be 1-63 characters long, and comply with [RFC1035] 1605 # (https://www.ietf.org/rfc/rfc1035.txt). 1606 # Specifically, the name must be 1-63 characters long and match the regular 1607 # expression `[a-z]([-a-z0-9]*[a-z0-9])?` which means the first 1608 # character must be a lowercase letter, and all following characters must be 1609 # a dash, lowercase letter, or digit, except the last character, which cannot 1610 # be a dash. 1611 "a_key": "A String", 1612 }, 1613 "serviceAccount": { # A Google Cloud Service Account. # The Google Cloud Service Account that will be used to access data and 1614 # services. By default, the compute service account associated with 1615 # `projectId` is used. 1616 "scopes": [ # List of scopes to be enabled for this service account on the VM. 1617 # The following scopes are automatically included: 1618 # 1619 # * https://www.googleapis.com/auth/compute 1620 # * https://www.googleapis.com/auth/devstorage.full_control 1621 # * https://www.googleapis.com/auth/genomics 1622 # * https://www.googleapis.com/auth/logging.write 1623 # * https://www.googleapis.com/auth/monitoring.write 1624 "A String", 1625 ], 1626 "email": "A String", # Email address of the service account. Defaults to `default`, 1627 # which uses the compute service account associated with the project. 1628 }, 1629 "clientId": "A String", # This field is deprecated. Use `labels` instead. Client-specified pipeline 1630 # operation identifier. 1631 "projectId": "A String", # Required. The project in which to run the pipeline. The caller must have 1632 # WRITER access to all Google Cloud services and resources (e.g. Google 1633 # Compute Engine) will be used. 1634 "keepVmAliveOnFailureDuration": "A String", # How long to keep the VM up after a failure (for example docker command 1635 # failed, copying input or output files failed, etc). While the VM is up, one 1636 # can ssh into the VM to debug. Default is 0; maximum allowed value is 1 day. 1637 "resources": { # The system resources for the pipeline run. # Specifies resource requirements/overrides for the pipeline run. 1638 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental 1639 # feature that may go away. Defaults to false. 1640 # Corresponds to `--no_address` flag for [gcloud compute instances create] 1641 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). 1642 # In order to use this, must be true for both create time and run time. 1643 # Cannot be true at run time if false at create time. If you need to ssh into 1644 # a private IP VM for debugging, you can ssh to a public VM and then ssh into 1645 # the private VM's Internal IP. If noAddress is set, this pipeline run may 1646 # only load docker images from Google Container Registry and not Docker Hub. 1647 # Before using this, you must 1648 # [configure access to Google services from internal 1649 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips). 1650 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB) 1651 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this, 1652 # must be true for both create time and run time. Cannot be true at run time 1653 # if false at create time. 1654 "disks": [ # Disks to attach. 1655 { # A Google Compute Engine disk resource specification. 1656 "name": "A String", # Required. The name of the disk that can be used in the pipeline 1657 # parameters. Must be 1 - 63 characters. 1658 # The name "boot" is reserved for system use. 1659 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB). 1660 # This field is not applicable for local SSD. 1661 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end 1662 # of the pipeline run, regardless of what this field is set to. 1663 "source": "A String", # The full or partial URL of the persistent disk to attach. See 1664 # https://cloud.google.com/compute/docs/reference/latest/instances#resource 1665 # and 1666 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots 1667 # for more details. 1668 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See 1669 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances 1670 # for more details. 1671 # Can only be set at create time. 1672 "mountPoint": "A String", # Required at create time and cannot be overridden at run time. 1673 # Specifies the path in the docker container where files on 1674 # this disk should be located. For example, if `mountPoint` 1675 # is `/mnt/disk`, and the parameter has `localPath` 1676 # `inputs/file.txt`, the docker container can access the data at 1677 # `/mnt/disk/inputs/file.txt`. 1678 "type": "A String", # Required. The type of the disk to create. 1679 }, 1680 ], 1681 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB). 1682 "zones": [ # List of Google Compute Engine availability zones to which resource 1683 # creation will restricted. If empty, any zone may be chosen. 1684 "A String", 1685 ], 1686 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1. 1687 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type. 1688 # By specifying this parameter, you will download and install the following 1689 # third-party software onto your managed Compute Engine instances: NVIDIA® 1690 # Tesla® drivers and NVIDIA® CUDA toolkit. 1691 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of 1692 # available accelerator types. 1693 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach. 1694 # By specifying this parameter, you will download and install the following 1695 # third-party software onto your managed Compute Engine instances: 1696 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit. 1697 }, 1698 }, 1699 "ephemeralPipeline": { # The pipeline object. Represents a transformation from a set of input # A new pipeline object to run once and then delete. 1700 # parameters to a set of output parameters. The transformation is defined 1701 # as a docker image and command to run within that image. Each pipeline 1702 # is run on a Google Compute Engine VM. A pipeline can be created with the 1703 # `create` method and then later run with the `run` method, or a pipeline can 1704 # be defined and run all at once with the `run` method. 1705 "pipelineId": "A String", # Unique pipeline id that is generated by the service when CreatePipeline 1706 # is called. Cannot be specified in the Pipeline used in the 1707 # CreatePipelineRequest, and will be populated in the response to 1708 # CreatePipeline and all subsequent Get and List calls. Indicates that the 1709 # service has registered this pipeline. 1710 "description": "A String", # User-specified description. 1711 "projectId": "A String", # Required. The project in which to create the pipeline. The caller must have 1712 # WRITE access. 1713 "outputParameters": [ # Output parameters of the pipeline. 1714 { # Parameters facilitate setting and delivering data into the 1715 # pipeline's execution environment. They are defined at create time, 1716 # with optional defaults, and can be overridden at run time. 1717 # 1718 # If `localCopy` is unset, then the parameter specifies a string that 1719 # is passed as-is into the pipeline, as the value of the environment 1720 # variable with the given name. A default value can be optionally 1721 # specified at create time. The default can be overridden at run time 1722 # using the inputs map. If no default is given, a value must be 1723 # supplied at runtime. 1724 # 1725 # If `localCopy` is defined, then the parameter specifies a data 1726 # source or sink, both in Google Cloud Storage and on the Docker container 1727 # where the pipeline computation is run. The service account associated with 1728 # the Pipeline (by 1729 # default the project's Compute Engine service account) must have access to the 1730 # Google Cloud Storage paths. 1731 # 1732 # At run time, the Google Cloud Storage paths can be overridden if a default 1733 # was provided at create time, or must be set otherwise. The pipeline runner 1734 # should add a key/value pair to either the inputs or outputs map. The 1735 # indicated data copies will be carried out before/after pipeline execution, 1736 # just as if the corresponding arguments were provided to `gsutil cp`. 1737 # 1738 # For example: Given the following `PipelineParameter`, specified 1739 # in the `inputParameters` list: 1740 # 1741 # ``` 1742 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 1743 # ``` 1744 # 1745 # where `disk` is defined in the `PipelineResources` object as: 1746 # 1747 # ``` 1748 # {name: "pd1", mountPoint: "/mnt/disk/"} 1749 # ``` 1750 # 1751 # We create a disk named `pd1`, mount it on the host VM, and map 1752 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 1753 # runtime, an entry for `input_file` would be required in the inputs 1754 # map, such as: 1755 # 1756 # ``` 1757 # inputs["input_file"] = "gs://my-bucket/bar.txt" 1758 # ``` 1759 # 1760 # This would generate the following gsutil call: 1761 # 1762 # ``` 1763 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 1764 # ``` 1765 # 1766 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 1767 # Docker container. Acceptable paths are: 1768 # 1769 # <table> 1770 # <thead> 1771 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 1772 # </thead> 1773 # <tbody> 1774 # <tr><td>file</td><td>file</td></tr> 1775 # <tr><td>glob</td><td>directory</td></tr> 1776 # </tbody> 1777 # </table> 1778 # 1779 # For outputs, the direction of the copy is reversed: 1780 # 1781 # ``` 1782 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 1783 # ``` 1784 # 1785 # Acceptable paths are: 1786 # 1787 # <table> 1788 # <thead> 1789 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 1790 # </thead> 1791 # <tbody> 1792 # <tr><td>file</td><td>file</td></tr> 1793 # <tr> 1794 # <td>file</td> 1795 # <td>directory - directory must already exist</td> 1796 # </tr> 1797 # <tr> 1798 # <td>glob</td> 1799 # <td>directory - directory will be created if it doesn't exist</td></tr> 1800 # </tbody> 1801 # </table> 1802 # 1803 # One restriction due to docker limitations, is that for outputs that are found 1804 # on the boot disk, the local path cannot be a glob and must be a file. 1805 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 1806 # If `localCopy` is present, then this must be a Google Cloud Storage path 1807 # beginning with `gs://`. 1808 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 1809 # `LocalCopy` indicates where on the VM the file should be. The value 1810 # given to this parameter (either at runtime or using `defaultValue`) 1811 # must be the remote path where the file should be. 1812 "path": "A String", # Required. The path within the user's docker container where 1813 # this input should be localized to and from, relative to the specified 1814 # disk's mount point. For example: file.txt, 1815 "disk": "A String", # Required. The name of the disk where this parameter is 1816 # located. Can be the name of one of the disks specified in the 1817 # Resources field, or "boot", which represents the Docker 1818 # instance's boot disk and has a mount point of `/`. 1819 }, 1820 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 1821 # as the key to the input and output maps in RunPipeline. 1822 "description": "A String", # Human-readable description. 1823 }, 1824 ], 1825 "inputParameters": [ # Input parameters of the pipeline. 1826 { # Parameters facilitate setting and delivering data into the 1827 # pipeline's execution environment. They are defined at create time, 1828 # with optional defaults, and can be overridden at run time. 1829 # 1830 # If `localCopy` is unset, then the parameter specifies a string that 1831 # is passed as-is into the pipeline, as the value of the environment 1832 # variable with the given name. A default value can be optionally 1833 # specified at create time. The default can be overridden at run time 1834 # using the inputs map. If no default is given, a value must be 1835 # supplied at runtime. 1836 # 1837 # If `localCopy` is defined, then the parameter specifies a data 1838 # source or sink, both in Google Cloud Storage and on the Docker container 1839 # where the pipeline computation is run. The service account associated with 1840 # the Pipeline (by 1841 # default the project's Compute Engine service account) must have access to the 1842 # Google Cloud Storage paths. 1843 # 1844 # At run time, the Google Cloud Storage paths can be overridden if a default 1845 # was provided at create time, or must be set otherwise. The pipeline runner 1846 # should add a key/value pair to either the inputs or outputs map. The 1847 # indicated data copies will be carried out before/after pipeline execution, 1848 # just as if the corresponding arguments were provided to `gsutil cp`. 1849 # 1850 # For example: Given the following `PipelineParameter`, specified 1851 # in the `inputParameters` list: 1852 # 1853 # ``` 1854 # {name: "input_file", localCopy: {path: "file.txt", disk: "pd1"}} 1855 # ``` 1856 # 1857 # where `disk` is defined in the `PipelineResources` object as: 1858 # 1859 # ``` 1860 # {name: "pd1", mountPoint: "/mnt/disk/"} 1861 # ``` 1862 # 1863 # We create a disk named `pd1`, mount it on the host VM, and map 1864 # `/mnt/pd1` to `/mnt/disk` in the docker container. At 1865 # runtime, an entry for `input_file` would be required in the inputs 1866 # map, such as: 1867 # 1868 # ``` 1869 # inputs["input_file"] = "gs://my-bucket/bar.txt" 1870 # ``` 1871 # 1872 # This would generate the following gsutil call: 1873 # 1874 # ``` 1875 # gsutil cp gs://my-bucket/bar.txt /mnt/pd1/file.txt 1876 # ``` 1877 # 1878 # The file `/mnt/pd1/file.txt` maps to `/mnt/disk/file.txt` in the 1879 # Docker container. Acceptable paths are: 1880 # 1881 # <table> 1882 # <thead> 1883 # <tr><th>Google Cloud storage path</th><th>Local path</th></tr> 1884 # </thead> 1885 # <tbody> 1886 # <tr><td>file</td><td>file</td></tr> 1887 # <tr><td>glob</td><td>directory</td></tr> 1888 # </tbody> 1889 # </table> 1890 # 1891 # For outputs, the direction of the copy is reversed: 1892 # 1893 # ``` 1894 # gsutil cp /mnt/disk/file.txt gs://my-bucket/bar.txt 1895 # ``` 1896 # 1897 # Acceptable paths are: 1898 # 1899 # <table> 1900 # <thead> 1901 # <tr><th>Local path</th><th>Google Cloud Storage path</th></tr> 1902 # </thead> 1903 # <tbody> 1904 # <tr><td>file</td><td>file</td></tr> 1905 # <tr> 1906 # <td>file</td> 1907 # <td>directory - directory must already exist</td> 1908 # </tr> 1909 # <tr> 1910 # <td>glob</td> 1911 # <td>directory - directory will be created if it doesn't exist</td></tr> 1912 # </tbody> 1913 # </table> 1914 # 1915 # One restriction due to docker limitations, is that for outputs that are found 1916 # on the boot disk, the local path cannot be a glob and must be a file. 1917 "defaultValue": "A String", # The default value for this parameter. Can be overridden at runtime. 1918 # If `localCopy` is present, then this must be a Google Cloud Storage path 1919 # beginning with `gs://`. 1920 "localCopy": { # LocalCopy defines how a remote file should be copied to and from the VM. # If present, this parameter is marked for copying to and from the VM. 1921 # `LocalCopy` indicates where on the VM the file should be. The value 1922 # given to this parameter (either at runtime or using `defaultValue`) 1923 # must be the remote path where the file should be. 1924 "path": "A String", # Required. The path within the user's docker container where 1925 # this input should be localized to and from, relative to the specified 1926 # disk's mount point. For example: file.txt, 1927 "disk": "A String", # Required. The name of the disk where this parameter is 1928 # located. Can be the name of one of the disks specified in the 1929 # Resources field, or "boot", which represents the Docker 1930 # instance's boot disk and has a mount point of `/`. 1931 }, 1932 "name": "A String", # Required. Name of the parameter - the pipeline runner uses this string 1933 # as the key to the input and output maps in RunPipeline. 1934 "description": "A String", # Human-readable description. 1935 }, 1936 ], 1937 "docker": { # The Docker execuctor specification. # Specifies the docker run information. 1938 "cmd": "A String", # Required. The command or newline delimited script to run. The command 1939 # string will be executed within a bash shell. 1940 # 1941 # If the command exits with a non-zero exit code, output parameter 1942 # de-localization will be skipped and the pipeline operation's 1943 # `error` field will be populated. 1944 # 1945 # Maximum command string length is 16384. 1946 "imageName": "A String", # Required. Image name from either Docker Hub or Google Container Registry. 1947 # Users that run pipelines must have READ access to the image. 1948 }, 1949 "resources": { # The system resources for the pipeline run. # Required. Specifies resource requirements for the pipeline run. 1950 # Required fields: 1951 # 1952 # * 1953 # minimumCpuCores 1954 # 1955 # * 1956 # minimumRamGb 1957 "noAddress": True or False, # Whether to assign an external IP to the instance. This is an experimental 1958 # feature that may go away. Defaults to false. 1959 # Corresponds to `--no_address` flag for [gcloud compute instances create] 1960 # (https://cloud.google.com/sdk/gcloud/reference/compute/instances/create). 1961 # In order to use this, must be true for both create time and run time. 1962 # Cannot be true at run time if false at create time. If you need to ssh into 1963 # a private IP VM for debugging, you can ssh to a public VM and then ssh into 1964 # the private VM's Internal IP. If noAddress is set, this pipeline run may 1965 # only load docker images from Google Container Registry and not Docker Hub. 1966 # Before using this, you must 1967 # [configure access to Google services from internal 1968 # IPs](https://cloud.google.com/compute/docs/configure-private-google-access#configuring_access_to_google_services_from_internal_ips). 1969 "minimumRamGb": 3.14, # The minimum amount of RAM to use. Defaults to 3.75 (GB) 1970 "preemptible": True or False, # Whether to use preemptible VMs. Defaults to `false`. In order to use this, 1971 # must be true for both create time and run time. Cannot be true at run time 1972 # if false at create time. 1973 "disks": [ # Disks to attach. 1974 { # A Google Compute Engine disk resource specification. 1975 "name": "A String", # Required. The name of the disk that can be used in the pipeline 1976 # parameters. Must be 1 - 63 characters. 1977 # The name "boot" is reserved for system use. 1978 "sizeGb": 42, # The size of the disk. Defaults to 500 (GB). 1979 # This field is not applicable for local SSD. 1980 "autoDelete": True or False, # Deprecated. Disks created by the Pipelines API will be deleted at the end 1981 # of the pipeline run, regardless of what this field is set to. 1982 "source": "A String", # The full or partial URL of the persistent disk to attach. See 1983 # https://cloud.google.com/compute/docs/reference/latest/instances#resource 1984 # and 1985 # https://cloud.google.com/compute/docs/disks/persistent-disks#snapshots 1986 # for more details. 1987 "readOnly": True or False, # Specifies how a sourced-base persistent disk will be mounted. See 1988 # https://cloud.google.com/compute/docs/disks/persistent-disks#use_multi_instances 1989 # for more details. 1990 # Can only be set at create time. 1991 "mountPoint": "A String", # Required at create time and cannot be overridden at run time. 1992 # Specifies the path in the docker container where files on 1993 # this disk should be located. For example, if `mountPoint` 1994 # is `/mnt/disk`, and the parameter has `localPath` 1995 # `inputs/file.txt`, the docker container can access the data at 1996 # `/mnt/disk/inputs/file.txt`. 1997 "type": "A String", # Required. The type of the disk to create. 1998 }, 1999 ], 2000 "bootDiskSizeGb": 42, # The size of the boot disk. Defaults to 10 (GB). 2001 "zones": [ # List of Google Compute Engine availability zones to which resource 2002 # creation will restricted. If empty, any zone may be chosen. 2003 "A String", 2004 ], 2005 "minimumCpuCores": 42, # The minimum number of cores to use. Defaults to 1. 2006 "acceleratorType": "A String", # Optional. The Compute Engine defined accelerator type. 2007 # By specifying this parameter, you will download and install the following 2008 # third-party software onto your managed Compute Engine instances: NVIDIA® 2009 # Tesla® drivers and NVIDIA® CUDA toolkit. 2010 # Please see https://cloud.google.com/compute/docs/gpus/ for a list of 2011 # available accelerator types. 2012 "acceleratorCount": "A String", # Optional. The number of accelerators of the specified type to attach. 2013 # By specifying this parameter, you will download and install the following 2014 # third-party software onto your managed Compute Engine instances: 2015 # NVIDIA® Tesla® drivers and NVIDIA® CUDA toolkit. 2016 }, 2017 "name": "A String", # Required. A user specified pipeline name that does not have to be unique. 2018 # This name can be used for filtering Pipelines in ListPipelines. 2019 }, 2020 "pipelineId": "A String", # The already created pipeline to run. 2021 } 2022 2023 x__xgafv: string, V1 error format. 2024 Allowed values 2025 1 - v1 error format 2026 2 - v2 error format 2027 2028Returns: 2029 An object of the form: 2030 2031 { # This resource represents a long-running operation that is the result of a 2032 # network API call. 2033 "metadata": { # An OperationMetadata or Metadata object. This will always be returned with the Operation. 2034 "a_key": "", # Properties of the object. Contains field @type with type URL. 2035 }, 2036 "done": True or False, # If the value is `false`, it means the operation is still in progress. 2037 # If `true`, the operation is completed, and either `error` or `response` is 2038 # available. 2039 "response": { # An Empty object. 2040 "a_key": "", # Properties of the object. Contains field @type with type URL. 2041 }, 2042 "name": "A String", # The server-assigned name, which is only unique within the same service that originally returns it. For example: `operations/CJHU7Oi_ChDrveSpBRjfuL-qzoWAgEw` 2043 "error": { # The `Status` type defines a logical error model that is suitable for # The error result of the operation in case of failure or cancellation. 2044 # different programming environments, including REST APIs and RPC APIs. It is 2045 # used by [gRPC](https://github.com/grpc). Each `Status` message contains 2046 # three pieces of data: error code, error message, and error details. 2047 # 2048 # You can find out more about this error model and how to work with it in the 2049 # [API Design Guide](https://cloud.google.com/apis/design/errors). 2050 "message": "A String", # A developer-facing error message, which should be in English. Any 2051 # user-facing error message should be localized and sent in the 2052 # google.rpc.Status.details field, or localized by the client. 2053 "code": 42, # The status code, which should be an enum value of google.rpc.Code. 2054 "details": [ # A list of messages that carry the error details. There is a common set of 2055 # message types for APIs to use. 2056 { 2057 "a_key": "", # Properties of the object. Contains field @type with type URL. 2058 }, 2059 ], 2060 }, 2061 }</pre> 2062</div> 2063 2064<div class="method"> 2065 <code class="details" id="setOperationStatus">setOperationStatus(body, x__xgafv=None)</code> 2066 <pre>Sets status of a given operation. Any new timestamps (as determined by 2067description) are appended to TimestampEvents. Should only be called by VMs 2068created by the Pipelines Service and not by end users. 2069 2070Args: 2071 body: object, The request body. (required) 2072 The object takes the form of: 2073 2074{ # Request to set operation status. Should only be used by VMs 2075 # created by the Pipelines Service and not by end users. 2076 "errorCode": "A String", 2077 "errorMessage": "A String", 2078 "timestampEvents": [ 2079 { # Stores the list of events and times they occured for major events in job 2080 # execution. 2081 "timestamp": "A String", # The time this event occured. 2082 "description": "A String", # String indicating the type of event 2083 }, 2084 ], 2085 "operationId": "A String", 2086 "validationToken": "A String", 2087 } 2088 2089 x__xgafv: string, V1 error format. 2090 Allowed values 2091 1 - v1 error format 2092 2 - v2 error format 2093 2094Returns: 2095 An object of the form: 2096 2097 { # A generic empty message that you can re-use to avoid defining duplicated 2098 # empty messages in your APIs. A typical example is to use it as the request 2099 # or the response type of an API method. For instance: 2100 # 2101 # service Foo { 2102 # rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); 2103 # } 2104 # 2105 # The JSON representation for `Empty` is empty JSON object `{}`. 2106 }</pre> 2107</div> 2108 2109</body></html>