1 /* 2 * Copyright (C) 2020 The Android Open Source Project 3 * 4 * Licensed under the Apache License, Version 2.0 (the "License"); 5 * you may not use this file except in compliance with the License. 6 * You may obtain a copy of the License at 7 * 8 * http://www.apache.org/licenses/LICENSE-2.0 9 * 10 * Unless required by applicable law or agreed to in writing, software 11 * distributed under the License is distributed on an "AS IS" BASIS, 12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 13 * See the License for the specific language governing permissions and 14 * limitations under the License. 15 */ 16 17 package android.hardware.camera2; 18 19 import android.annotation.NonNull; 20 21 import java.util.concurrent.Executor; 22 23 /** 24 * A camera capture session that enables access to device-specific camera extensions, which 25 * often use multi-frame bursts and sophisticated post-process algorithms for image capture. 26 * 27 * <p>The capture session will be returned after a successful call to 28 * {@link CameraDevice#createExtensionSession} as part of the argument 29 * in the registered state callback {@link StateCallback#onConfigured} 30 * method. </p> 31 * 32 * <p>Note that CameraExtensionSession is currently limited to a maximum of two output 33 * surfaces for continuous repeating and multi-frame processing respectively. Some 34 * features such as capture settings will not be supported as the device-specific 35 * Extension is allowed to override all capture parameters.</p> 36 * 37 * <p>Information about support for specific device-specific extensions can be queried 38 * from {@link CameraExtensionCharacteristics}. </p> 39 */ 40 public abstract class CameraExtensionSession implements AutoCloseable { 41 /** @hide */ CameraExtensionSession()42 public CameraExtensionSession () {} 43 44 /** 45 * A callback object for tracking the progress of a 46 * {@link CaptureRequest} submitted to the camera device. 47 * 48 * <p>This callback is invoked when a request triggers a capture to start, 49 * and when the device-specific Extension post processing begins. In case of an 50 * error capturing an image, the error method is triggered instead of 51 * the completion method.</p> 52 * 53 * @see #capture 54 * @see #setRepeatingRequest 55 */ 56 public static abstract class ExtensionCaptureCallback { 57 58 /** 59 * This method is called when the camera device has started 60 * capturing the initial input image of the device-specific extension 61 * post-process request. 62 * 63 * <p>This callback is invoked right as the capture of a frame begins, 64 * so it is the most appropriate time for playing a shutter sound, 65 * or triggering UI indicators of capture.</p> 66 * 67 * <p>The request that is being used for this capture is provided, 68 * along with the actual timestamp for the start of exposure.</p> 69 * 70 * <p>The default implementation of this method does nothing.</p> 71 * 72 * @param session the session received during 73 * {@link StateCallback#onConfigured(CameraExtensionSession)} 74 * @param request the request for the capture that just begun 75 * @param timestamp the timestamp at start of capture for repeating 76 * request or the timestamp at start of capture of the 77 * first frame in a multi-frame capture. 78 */ onCaptureStarted(@onNull CameraExtensionSession session, @NonNull CaptureRequest request, long timestamp)79 public void onCaptureStarted(@NonNull CameraExtensionSession session, 80 @NonNull CaptureRequest request, long timestamp) { 81 // default empty implementation 82 } 83 84 /** 85 * This method is called when an image (or images in case of multi-frame 86 * capture) is captured and device-specific extension 87 * processing is triggered. 88 * 89 * <p>Each request will generate at most {@code 1} 90 * {@link #onCaptureProcessStarted}.</p> 91 * 92 * <p>The default implementation of this method does nothing.</p> 93 * 94 * @param session the session received during 95 * {@link StateCallback#onConfigured(CameraExtensionSession)} 96 * @param request The request that was given to the CameraExtensionSession 97 * 98 * @see #capture 99 * @see #setRepeatingRequest 100 */ onCaptureProcessStarted(@onNull CameraExtensionSession session, @NonNull CaptureRequest request)101 public void onCaptureProcessStarted(@NonNull CameraExtensionSession session, 102 @NonNull CaptureRequest request) { 103 // default empty implementation 104 } 105 106 /** 107 * This method is called instead of 108 * {@link #onCaptureProcessStarted} when the camera device failed 109 * to produce the required input for the device-specific extension. The 110 * cause could be a failed camera capture request, a failed 111 * capture result or dropped camera frame. 112 * 113 * <p>Other requests are unaffected, and some or all image buffers 114 * from the capture may have been pushed to their respective output 115 * streams.</p> 116 * 117 * <p>The default implementation of this method does nothing.</p> 118 * 119 * @param session the session received during 120 * {@link StateCallback#onConfigured(CameraExtensionSession)} 121 * @param request The request that was given to the CameraDevice 122 * 123 * @see #capture 124 * @see #setRepeatingRequest 125 */ onCaptureFailed(@onNull CameraExtensionSession session, @NonNull CaptureRequest request)126 public void onCaptureFailed(@NonNull CameraExtensionSession session, 127 @NonNull CaptureRequest request) { 128 // default empty implementation 129 } 130 131 /** 132 * This method is called independently of the others in 133 * ExtensionCaptureCallback, when a capture sequence finishes. 134 * 135 * <p>In total, there will be at least one 136 * {@link #onCaptureProcessStarted}/{@link #onCaptureFailed} 137 * invocation before this callback is triggered. If the capture 138 * sequence is aborted before any requests have begun processing, 139 * {@link #onCaptureSequenceAborted} is invoked instead.</p> 140 * 141 * <p>The default implementation does nothing.</p> 142 * 143 * @param session the session received during 144 * {@link StateCallback#onConfigured(CameraExtensionSession)} 145 * @param sequenceId A sequence ID returned by the {@link #capture} 146 * family of functions. 147 * @see #onCaptureSequenceAborted 148 */ onCaptureSequenceCompleted(@onNull CameraExtensionSession session, int sequenceId)149 public void onCaptureSequenceCompleted(@NonNull CameraExtensionSession session, 150 int sequenceId) { 151 // default empty implementation 152 } 153 154 /** 155 * This method is called when a capture sequence aborts. 156 * 157 * <p>Due to the asynchronous nature of the camera device, not all 158 * submitted captures are immediately processed. It is possible to 159 * clear out the pending requests by a variety of operations such 160 * as {@link CameraExtensionSession#stopRepeating}. When such an event 161 * happens, {@link #onCaptureProcessStarted} will not be called.</p> 162 * 163 * <p>The default implementation does nothing.</p> 164 * 165 * @param session the session received during 166 * {@link StateCallback#onConfigured(CameraExtensionSession)} 167 * @param sequenceId A sequence ID returned by the {@link #capture} 168 * family of functions. 169 * @see #onCaptureProcessStarted 170 */ onCaptureSequenceAborted(@onNull CameraExtensionSession session, int sequenceId)171 public void onCaptureSequenceAborted(@NonNull CameraExtensionSession session, 172 int sequenceId) { 173 // default empty implementation 174 } 175 176 /** 177 * This method is called when an image capture has fully completed and all the 178 * result metadata is available. 179 * 180 * <p>This callback will only be called in case 181 * {@link CameraExtensionCharacteristics#getAvailableCaptureResultKeys} returns a valid 182 * non-empty list.</p> 183 * 184 * <p>The default implementation of this method does nothing.</p> 185 * 186 * @param session The session received during 187 * {@link StateCallback#onConfigured(CameraExtensionSession)} 188 * @param request The request that was given to the CameraDevice 189 * @param result The total output metadata from the capture, which only includes the 190 * capture result keys advertised as supported in 191 * {@link CameraExtensionCharacteristics#getAvailableCaptureResultKeys}. 192 * 193 * @see #capture 194 * @see #setRepeatingRequest 195 * @see CameraExtensionCharacteristics#getAvailableCaptureResultKeys 196 */ onCaptureResultAvailable(@onNull CameraExtensionSession session, @NonNull CaptureRequest request, @NonNull TotalCaptureResult result)197 public void onCaptureResultAvailable(@NonNull CameraExtensionSession session, 198 @NonNull CaptureRequest request, @NonNull TotalCaptureResult result) { 199 // default empty implementation 200 } 201 } 202 203 /** 204 * A callback object for receiving updates about the state of a camera extension session. 205 * 206 */ 207 public static abstract class StateCallback { 208 209 /** 210 * This method is called when the camera device has finished configuring itself, and the 211 * session can start processing capture requests. 212 * 213 * <p>If the camera device configuration fails, then {@link #onConfigureFailed} will 214 * be invoked instead of this callback.</p> 215 * 216 * @param session A valid extension session 217 */ onConfigured(@onNull CameraExtensionSession session)218 public abstract void onConfigured(@NonNull CameraExtensionSession session); 219 220 /** 221 * This method is called if the session cannot be configured as requested. 222 * 223 * <p>This can happen if the set of requested outputs contains unsupported sizes, 224 * too many outputs are requested at once or when trying to initialize multiple 225 * concurrent extension sessions from two (or more) separate camera devices 226 * or the camera device encounters an unrecoverable error during configuration.</p> 227 * 228 * <p>The session is considered to be closed, and all methods called on it after this 229 * callback is invoked will throw an IllegalStateException.</p> 230 * 231 * @param session the session instance that failed to configure 232 */ onConfigureFailed(@onNull CameraExtensionSession session)233 public abstract void onConfigureFailed(@NonNull CameraExtensionSession session); 234 235 /** 236 * This method is called when the session is closed. 237 * 238 * <p>A session is closed when a new session is created by the parent camera device, 239 * or when the parent camera device is closed (either by the user closing the device, 240 * or due to a camera device disconnection or fatal error).</p> 241 * 242 * <p>Once a session is closed, all methods on it will throw an IllegalStateException, and 243 * any repeating requests are stopped (as if {@link #stopRepeating()} was called). 244 * However, any in-progress capture requests submitted to the session will be completed 245 * as normal.</p> 246 * 247 * @param session the session received during 248 * {@link StateCallback#onConfigured(CameraExtensionSession)} 249 */ onClosed(@onNull CameraExtensionSession session)250 public void onClosed(@NonNull CameraExtensionSession session) { 251 // default empty implementation 252 } 253 } 254 255 /** 256 * Get the camera device that this session is created for. 257 */ 258 @NonNull getDevice()259 public android.hardware.camera2.CameraDevice getDevice() { 260 throw new UnsupportedOperationException("Subclasses must override this method"); 261 } 262 263 /** 264 * Submit a request for device-specific processing using input 265 * from the camera device, to produce a single high-quality output result. 266 * 267 * <p>Note that single capture requests currently do not support 268 * client parameters except for controls advertised in 269 * {@link CameraExtensionCharacteristics#getAvailableCaptureRequestKeys}. 270 * The rest of the settings included in the request will be entirely overridden by 271 * the device-specific extension. </p> 272 * 273 * <p>The {@link CaptureRequest.Builder#addTarget} supports only one 274 * ImageFormat.YUV_420_888 or ImageFormat.JPEG target surface. {@link CaptureRequest} 275 * arguments that include further targets will cause 276 * IllegalArgumentException to be thrown. </p> 277 * 278 * <p>Starting with Android {@link android.os.Build.VERSION_CODES#TIRAMISU} single capture 279 * requests will also support the preview {@link android.graphics.ImageFormat#PRIVATE} target 280 * surface. These can typically be used for enabling AF/AE triggers. Do note, that single 281 * capture requests referencing both output surfaces remain unsupported.</p> 282 * 283 * <p>Each request will produce one new frame for one target Surface, set 284 * with the CaptureRequest builder's 285 * {@link CaptureRequest.Builder#addTarget} method.</p> 286 * 287 * <p>Multiple requests can be in progress at once. Requests are 288 * processed in first-in, first-out order.</p> 289 * 290 * <p>Requests submitted through this method have higher priority than 291 * those submitted through {@link #setRepeatingRequest}, and will be 292 * processed as soon as the current repeat processing completes.</p> 293 * 294 * @param request the settings for this capture 295 * @param executor the executor which will be used for invoking the 296 * listener. 297 * @param listener The callback object to notify once this request has 298 * been processed. 299 * @return int A unique capture sequence ID used by 300 * {@link ExtensionCaptureCallback#onCaptureSequenceCompleted}. 301 * @throws CameraAccessException if the camera device is no longer 302 * connected or has encountered a fatal error 303 * @throws IllegalStateException if this session is no longer active, 304 * either because the session was explicitly closed, a new 305 * session has been created or the camera device has been 306 * closed. 307 * @throws IllegalArgumentException if the request targets no Surfaces 308 * or Surfaces that are not configured as outputs for this 309 * session; or the request targets a set of Surfaces that 310 * cannot be submitted simultaneously. 311 */ capture(@onNull CaptureRequest request, @NonNull Executor executor, @NonNull ExtensionCaptureCallback listener)312 public int capture(@NonNull CaptureRequest request, 313 @NonNull Executor executor, 314 @NonNull ExtensionCaptureCallback listener) throws CameraAccessException { 315 throw new UnsupportedOperationException("Subclasses must override this method"); 316 } 317 318 /** 319 * Request endlessly repeating device-specific extension processing of 320 * camera images. 321 * 322 * <p>With this method, the camera device will continually capture images 323 * and process them using the device-specific extension at the maximum 324 * rate possible.</p> 325 * 326 * <p>Note that repeating capture requests currently do not support 327 * client parameters except for controls advertised in 328 * {@link CameraExtensionCharacteristics#getAvailableCaptureRequestKeys}. 329 * The rest of the settings included in the request will be entirely overridden by 330 * the device-specific extension. </p> 331 * 332 * <p>The {@link CaptureRequest.Builder#addTarget} supports only one 333 * target surface. {@link CaptureRequest} arguments that include further 334 * targets will cause IllegalArgumentException to be thrown.</p> 335 * 336 * <p>Repeating requests are a simple way for an application to maintain a 337 * preview or other continuous stream of frames.</p> 338 * 339 * <p>Repeat requests have lower priority than those submitted 340 * through {@link #capture}, so if {@link #capture} is called when a 341 * repeating request is active, the capture request will be processed 342 * before any further repeating requests are processed.</p> 343 * 344 * <p>To stop the repeating capture, call {@link #stopRepeating}.</p> 345 * 346 * <p>Calling this method will replace any earlier repeating request.</p> 347 * 348 * @param request the request to repeat indefinitely 349 * @param executor the executor which will be used for invoking the 350 * listener. 351 * @param listener The callback object to notify every time the 352 * request finishes processing. 353 * @return int A unique capture sequence ID used by 354 * {@link ExtensionCaptureCallback#onCaptureSequenceCompleted}. 355 * @throws CameraAccessException if the camera device is no longer 356 * connected or has encountered a fatal error 357 * @throws IllegalStateException if this session is no longer active, 358 * either because the session was explicitly closed, a new 359 * session has been created or the camera device has been 360 * closed. 361 * @throws IllegalArgumentException If the request references no 362 * Surfaces or references Surfaces that are not currently 363 * configured as outputs. 364 * @see #capture 365 */ setRepeatingRequest(@onNull CaptureRequest request, @NonNull Executor executor, @NonNull ExtensionCaptureCallback listener)366 public int setRepeatingRequest(@NonNull CaptureRequest request, 367 @NonNull Executor executor, 368 @NonNull ExtensionCaptureCallback listener) throws CameraAccessException { 369 throw new UnsupportedOperationException("Subclasses must override this method"); 370 } 371 372 /** 373 * Cancel any ongoing repeating capture set by 374 * {@link #setRepeatingRequest setRepeatingRequest}. Has no effect on 375 * requests submitted through {@link #capture capture}. 376 * 377 * <p>Any currently in-flight captures will still complete.</p> 378 * 379 * @throws CameraAccessException if the camera device is no longer 380 * connected or has encountered a fatal error 381 * @throws IllegalStateException if this session is no longer active, 382 * either because the session was explicitly closed, a new 383 * session has been created or the camera device has been closed. 384 * @see #setRepeatingRequest 385 */ stopRepeating()386 public void stopRepeating() throws CameraAccessException { 387 throw new UnsupportedOperationException("Subclasses must override this method"); 388 } 389 390 /** 391 * Close this capture session asynchronously. 392 * 393 * <p>Closing a session frees up the target output Surfaces of the session 394 * for reuse with either a new session, or to other APIs that can draw 395 * to Surfaces.</p> 396 * 397 * <p>Note that creating a new capture session with 398 * {@link android.hardware.camera2.CameraDevice#createCaptureSession} or 399 * {@link android.hardware.camera2.CameraDevice#createExtensionSession} 400 * will close any existing capture session automatically, and call the 401 * older session listener's {@link StateCallback#onClosed} callback. 402 * Using 403 * {@link android.hardware.camera2.CameraDevice#createCaptureSession} or 404 * {@link android.hardware.camera2.CameraDevice#createExtensionSession} 405 * directly without closing is the recommended approach for quickly 406 * switching to a new session, since unchanged target outputs can be 407 * reused more efficiently.</p> 408 * 409 * <p>Once a session is closed, all methods on it will throw an 410 * IllegalStateException, and any repeating requests are 411 * stopped (as if {@link #stopRepeating()} was called).</p> 412 * 413 * <p>Closing a session is idempotent; closing more than once has no 414 * effect.</p> 415 */ close()416 public void close() throws CameraAccessException { 417 throw new UnsupportedOperationException("Subclasses must override this method"); 418 } 419 } 420