Playing videos and music is a popular activity on Android devices. The Android framework provides {@link android.media.MediaPlayer} as a quick solution for playing media with minimal code, and the {@link android.media.MediaCodec} and {@link android.media.MediaExtractor} classes are provided for building custom media players. The open source project, ExoPlayer, is a solution between these two options, providing a pre-built player that you can extend.
ExoPlayer supports features not currently provided by {@link android.media.MediaPlayer}, including Dynamic adaptive streaming over HTTP (DASH), SmoothStreaming, and persistent caching. ExoPlayer can be extended to handle additional media formats, and because you include it as part of your app code, you can update it along with your app.
This guide describes how to use ExoPlayer for playing Android supported media formats, as well as DASH and SmoothStreaming playback. This guide also discusses ExoPlayer events, messages, DRM support and guidelines for customizing the player.
Note: ExoPlayer is an open source project that is not part of the Android framework and is distributed separately from the Android SDK. The project contains a library and a demo app that shows both simple and more advanced use of ExoPlayer:
ExoPlayer is a media player built on top of the {@link android.media.MediaExtractor} and {@link android.media.MediaCodec} APIs released in Android 4.1 (API level 16). At the core of this library is the {@code ExoPlayer} class. This class maintains the player’s global state, but makes few assumptions about the nature of the media being played, such as how the media data is obtained, how it is buffered or its format. You inject this functionality through ExoPlayer’s {@code prepare()} method in the form of {@code TrackRenderer} objects.
ExoPlayer provides default {@code TrackRenderer} implementations for audio and video, which make use of the {@link android.media.MediaCodec} and {@link android.media.AudioTrack} classes in the Android framework. Both renderers require a {@code SampleSource} object, from which they obtain individual media samples for playback. Figure 1 shows the high level object model for an ExoPlayer implementation configured to play audio and video using these components.
Figure 1. High level object model for an ExoPlayer configured to play audio and video using {@code TrackRenderer} objects
A {@code TrackRenderer} processes a component of media for playback, such as video, audio or text. The ExoPlayer class invokes methods on its {@code TrackRenderer} instances from a single playback thread, and by doing so causes each media component to be rendered as the global playback position is advanced. The ExoPlayer library provides {@code MediaCodecVideoTrackRenderer} as the default implementations rendering video and {@code MediaCodecAudioTrackRenderer} for audio. Both implementations make use of {@link android.media.MediaCodec} to decode individual media samples. They can handle all audio and video formats supported by a given Android device (see Supported Media Formats for details). The ExoPlayer library also provides an implementation for rendering text called {@code TextTrackRenderer}.
The code example below outlines the main steps required to instantiate an ExoPlayer to play video and audio using the standard {@code TrackRenderer} implementations.
// 1. Instantiate the player.
player = ExoPlayer.Factory.newInstance(RENDERER_COUNT);
// 2. Construct renderers.
MediaCodecVideoTrackRenderer videoRenderer = …
MediaCodecAudioTrackRenderer audioRenderer = ...
// 3. Inject the renderers through prepare.
player.prepare(videoRenderer, audioRenderer);
// 4. Pass the surface to the video renderer.
player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE,
surface);
// 5. Start playback.
player.setPlayWhenReady(true);
...
player.release(); // Don’t forget to release when done!
For a complete example, see the {@code SimplePlayerActivity} in the ExoPlayer demo app, which correctly manages an ExoPlayer instance with respect to both the {@link android.app.Activity} and {@link android.view.Surface} lifecycles.
A standard {@code TrackRenderer} implementation requires a {@code SampleSource} to be provided in its constructor. A {@code SampleSource} object provides format information and media samples to be rendered. The ExoPlayer library provides {@code FrameworkSampleSource} and {@code ChunkSampleSource}. The {@code FrameworkSampleSource} class uses {@link android.media.MediaExtractor} to request, buffer and extract the media samples. The {@code ChunkSampleSource} class provides adaptive playback using DASH or SmoothStreaming, and implements networking, buffering and media extraction within the ExoPlayer library.
In order to render media formats supported by the Android framework, the {@code FrameworkSampleSource} class uses {@link android.media.MediaExtractor} for networking, buffering and sample extraction functionality. By doing so, it supports any media container format supported by the version of Android where it is running. For more information about media formats supported by Android, see Supported Media Formats.
The diagram in Figure 2 shows the object model for an ExoPlayer implementation using {@code FrameworkSampleSource}.
Figure 2. Object model for an implementation of ExoPlayer that renders media formats supported by Android using {@code FrameworkSampleSource}
The following code example outlines how the video and audio renderers are constructed to load the video from a specified URI.
FrameworkSampleSource sampleSource = new FrameworkSampleSource(
activity, uri, null, 2);
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(
sampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT, 0,
mainHandler, playerActivity, 50);
MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(
sampleSource, null, true);
The ExoPlayer demo app provides a complete implementation of this code in {@code DefaultRendererBuilder}. The {@code SimplePlaybackActivity} class uses it to play one of the videos available in the demo app. Note that in the example, video and audio are muxed, meaning they are streamed together from a single URI. The {@code FrameworkSampleSource} instance provides video samples to the {@code videoRenderer} object and audio samples to the {@code audioRenderer} object as they are extracted from the media container format. It is also possible to play demuxed media, where video and audio are streamed separately from different URIs. This functionality can be achieved by having two {@code FrameworkSampleSource} instances instead of one.
ExoPlayer supports adaptive streaming, which allows the quality of the media data to be adjusted during playback based on the network conditions. DASH and SmoothStreaming are examples of adaptive streaming technologies. Both these approaches load media in small chunks (typically 2 to 10 seconds in duration). Whenever a chunk of media is requested, the client selects from a number of possible formats. For example, a client may select a high quality format if network conditions are good, or a low quality format if network conditions are bad. In both techniques, video and audio are streamed separately.
ExoPlayer supports adaptive playback through use of the {@code ChunkSampleSource} class, which loads chunks of media data from which individual samples can be extracted. Each {@code ChunkSampleSource} requires a {@code ChunkSource} object to be injected through its constructor, which is responsible for providing media chunks from which to load and read samples. The {@code DashMp4ChunkSource} and {@code SmoothStreamingChunkSource} classes provide DASH and SmoothStreaming playback using the FMP4 container format. The {@code DashWebMChunkSource} class uses the WebM container format to provide DASH playback.
All of the standard {@code ChunkSource} implementations require a {@code FormatEvaluator} and a {@code DataSource} to be injected through their constructors. The {@code FormatEvaluator} objects select from the available formats before each chunk is loaded. The {@code DataSource} objects are responsible for actually loading the data. Finally, the {@code ChunkSampleSources} require a {@code LoadControl} object that controls the chunk buffering policy.
The object model of an ExoPlayer configured for a DASH adaptive playback is shown in the diagram below. This example uses an {@code HttpDataSource} object to stream the media over the network. The video quality is varied at runtime using the adaptive implementation of {@code FormatEvaluator}, while audio is played at a fixed quality level.
Figure 3. Object model for a DASH adaptive playback using ExoPlayer
The following code example outlines how the video and audio renderers are constructed.
Handler mainHandler = playerActivity.getMainHandler();
LoadControl loadControl = new DefaultLoadControl(
new BufferPool(BUFFER_SEGMENT_SIZE));
BandwidthMeter bandwidthMeter = new BandwidthMeter();
// Build the video renderer.
DataSource videoDataSource = new HttpDataSource(userAgent,
HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter);
ChunkSource videoChunkSource = new DashMp4ChunkSource(videoDataSource,
new AdaptiveEvaluator(bandwidthMeter), videoRepresentations);
ChunkSampleSource videoSampleSource = new ChunkSampleSource(videoChunkSource,
loadControl, VIDEO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE, true);
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(
videoSampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT,
0, mainHandler, playerActivity, 50);
// Build the audio renderer.
DataSource audioDataSource = new HttpDataSource(userAgent,
HttpDataSource.REJECT_PAYWALL_TYPES, bandwidthMeter);
ChunkSource audioChunkSource = new DashMp4ChunkSource(audioDataSource,
new FormatEvaluator.FixedEvaluator(), audioRepresentation);
SampleSource audioSampleSource = new ChunkSampleSource(audioChunkSource,
loadControl, AUDIO_BUFFER_SEGMENTS * BUFFER_SEGMENT_SIZE, true);
MediaCodecAudioTrackRenderer audioRenderer = new MediaCodecAudioTrackRenderer(
audioSampleSource, null, true);
In this code, {@code videoRepresentations} and {@code audioRepresentation} are {@code Representation} objects, each of which describes one of the available media streams. In the DASH model, these streams are parsed from a media presentation description (MPD) file. The ExoPlayer library provides a {@code MediaPresentationDescriptionParser} class to obtain {@code Representation} objects from MPD files.
Note: Building Representation objects from MPD files is not required. You can build Representation objects from other data sources if necessary.
The ExoPlayer demo app provides complete implementation of this code in {@code DashVodRendererBuilder}. The {@code SimplePlaybackActivity} class uses this builder to construct renderers for playing DASH sample videos in the demo app. It asynchronously fetches a specified MPD file in order to construct the required {@code Representation} objects. For an equivalent SmoothStreaming example, see the {@code SmoothStreamingRendererBuilder} class in the demo app.
For DASH and SmoothStreaming playback, consider both static format selection at the start of playback and dynamic format selection during playback. Static format selection should be used to filter out formats that should not be used throughout the playback, for example formats with resolutions higher than the maximum supported by the playback device. Dynamic selection varies the selected format during playback, typically to adapt video quality in response to changes in network conditions.
When preparing a player, you should consider filtering out some of the available formats if they are not useable for playback. Static format selection allows you to filter out formats that cannot be used on a particular device or are not compatible with your player. For audio playback, this often means picking a single format to play and discarding the others.
For video playback, filtering formats can be more complicated. Apps should first eliminate any streams that whose resolution is too high to be played by the device. For H.264, which is normally used for DASH and SmoothStreaming playback, ExoPlayer’s {@code MediaCodecUtil} class provides a {@code maxH264DecodableFrameSize()} method that can be used to determine what resolution streams the device is able to handle, as shown in the following code example:
int maxDecodableFrameSize = MediaCodecUtil.maxH264DecodableFrameSize();
Format format = representation.format;
if (format.width * format.height <= maxDecodableFrameSize) {
// The device can play this stream.
videoRepresentations.add(representation);
} else {
// The device isn't capable of playing this stream.
}
This approach is used to filter {@code Representations} in the {@code DashVodRendererBuilder} class of the ExoPlayer demo app, and similarly to filter track indices in {@code SmoothStreamingRendererBuilder}.
In addition to eliminating unsupported formats, it should be noted that the ability to seamlessly switch between H.264 streams of different resolution is an optional decoder feature available in Android 4.3 (API level 16) and higher, and so is not supported by all devices. The availability of an adaptive H.264 decoder can be queried using {@code MediaCodecUtil}, as shown in the following code example:
boolean isAdaptive = MediaCodecUtil.getDecoderInfo(MimeTypes.VIDEO_H264).adaptive;
The {@code MediaCodecVideoTrackRenderer} class is still able to handle resolution changes on devices that do not have adaptive decoders, however the switch is not seamless. Typically, the switch creates a small discontinuity in visual output lasting around 50-100ms. For devices that do not provide an adaptive decoder, app developers may choose to adapt between formats at a single fixed resolution so as to avoid discontinuities. The ExoPlayer demo app implementation does not pick a fixed resolution.
During playback, you can use a {@code FormatEvaluator} to dynamically select from the available video formats. The ExoPlayer library provides a {@code FormatEvaluator.Adaptive} implementation for dynamically selecting between video formats based on the current network conditions.
This class provides a simple, general purpose reference implementation, however you are encouraged to write your own {@code FormatEvaluator} implementation to best suit your particular needs.
During playback, your app can listen for events generated by the ExoPlayer that indicate the overall state of the player. These events are useful as triggers for updating the app user interface such as playback controls. Many ExoPlayer components also report their own component specific low level events, which can be useful for performance monitoring.
ExoPlayer allows instances of {@code ExoPlayer.Listener} to be added and removed using its {@code addListener()} and {@code removeListener()} methods. Registered listeners are notified of changes in playback state, as well as when errors occur that cause playback to fail. For more information about the valid playback states and the possible transitions between them, see the ExoPlayer source code.
Developers who implement custom playback controls should register a listener and use it to update their controls as the player’s state changes. An app should also show an appropriate error to the user if playback fails.
In addition to high level listeners, many of the individual components provided by the ExoPlayer library allow their own event listeners. For example, {@code MediaCodecVideoTrackRenderer} has constructors that take a {@code MediaCodecVideoTrackRenderer.EventListener}. In the ExoPlayer demo app, {@code SimplePlayerActivity} acts as a listener so that it can adjust the dimensions of the target surface to have the correct height and width ratio for the video being played:
@Override
public void onVideoSizeChanged(int width, int height) {
surfaceView.setVideoWidthHeightRatio(height == 0 ? 1 : (float) width / height);
}
The {@code RendererBuilder} classes in the ExoPlayer demo app inject the activity as the listener, for example in the {@code DashVodRendererBuilder} class:
MediaCodecVideoTrackRenderer videoRenderer = new MediaCodecVideoTrackRenderer(
videoSampleSource, null, true, MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT,
0, mainHandler, playerActivity, 50);
Note that you must pass a {@link android.os.Handler} object to the renderer, which determines the thread on which the listener’s methods are invoked. In most cases, you should use a {@link android.os.Handler} associated with the app’s main thread, as is the case in this example.
Listening to individual components can be useful for adjusting UI based on player events, as in the example above. Listening to component events can also be helpful for logging performance metrics. For example, {@code MediaCodecVideoTrackRenderer} notifies its listener of dropped video frames. A developer may wish to log such metrics to track playback performance in their app.
Many components also notify their listeners when errors occur. Such errors may or may not cause playback to fail. If an error does not cause playback to fail, it may still result in degraded performance, and so you may wish to log all errors in order to track playback performance. Note that an ExoPlayer instance always notifies its high level listeners of errors that cause playback to fail, in addition to the listener of the individual component from which the error originated. Hence, you should display error messages to users only from high level listeners. Within individual component listeners, you should use error notifications only for informational purposes.
Some ExoPlayer components allow changes in configuration during playback. By convention, you make these changes by passing asynchronous messages through the ExoPlayer to the component. This approach ensures both thread safety and that the configuration change is executed in order with any other operations being performed on the player.
The most common use of messaging is passing a target surface to {@code MediaCodecVideoTrackRenderer}:
player.sendMessage(videoRenderer, MediaCodecVideoTrackRenderer.MSG_SET_SURFACE,
surface);
Note that if the surface needs to be cleared because {@link android.view.SurfaceHolder.Callback#surfaceDestroyed SurfaceHolder.Callback.surfaceDestroyed()} has been invoked, then you must send this message using the blocking variant of {@code sendMessage()}:
player.blockingSendMessage(videoRenderer,
MediaCodecVideoTrackRenderer.MSG_SET_SURFACE, null);
You must use a blocking message because the contract of {@link android.view.SurfaceHolder.Callback#surfaceDestroyed surfaceDestroyed()} requires that the app does not attempt to access the surface after the method returns. The {@code SimplePlayerActivity} class in the demo app demonstrates how the surface should be set and cleared.
One of the main benefits of ExoPlayer over {@link android.media.MediaPlayer} is the ability to customize and extend the player to better suit the developer’s use case. The ExoPlayer library is designed specifically with this in mind, defining a number of abstract base classes and interfaces that make it possible for app developers to easily replace the default implementations provided by the library. Here are some use cases for building custom components:
If a custom component needs to report events back to the app, we recommend that you do so using the same model as existing ExoPlayer components, where an event listener is passed together with a {@link android.os.Handler} to the constructor of the component.
We recommended that custom components use the same model as existing ExoPlayer components to allow reconfiguration by the app during playback, as described in Sending messages to components. To do this, you should implement a {@code ExoPlayerComponent} and receive configuration changes in its {@code handleMessage()} method. Your app should pass configuration changes by calling ExoPlayer’s {@code sendMessage()} and {@code blockingSendMessage()} methods.
On Android 4.3 (API level 18) and higher, ExoPlayer supports Digital Rights Managment (DRM) protected playback. In order to play DRM protected content with ExoPlayer, your app must inject a {@code DrmSessionManager} into the {@code MediaCodecVideoTrackRenderer} and {@code MediaCodecAudioTrackRenderer} constructors. A {@code DrmSessionManager} object is responsible for providing the {@code MediaCrypto} object required for decryption, as well as ensuring that the required decryption keys are available to the underlying DRM module being used.
The ExoPlayer library provides a default implementation of {@code DrmSessionManager}, called {@code StreamingDrmSessionManager}, which uses {@link android.media.MediaDrm}. The session manager supports any DRM scheme for which a modular DRM component exists on the device. All Android devices are required to support Widevine modular DRM (with L3 security, although many devices also support L1). Some devices may support additional schemes such as PlayReady.
The {@code StreamingDrmSessionManager} class requires a {@code MediaDrmCallback} to be injected into its constructor, which is responsible for actually making provisioning and key requests. You should implement this interface to make network requests to your license server and obtain the required keys. The {@code WidevineTestMediaDrmCallback} class in the ExoPlayer demo app sends requests to a Widevine test server.