1# Taking Over the Media Playback on Web Pages 2 3The **Web** component provides the capability for applications to take over media playback on web pages, which improves media playback qualities on the web page. 4 5## When to Use 6 7On web pages, media player one provides limited features, the resolution of media playback is low, and some videos cannot be played. 8 9Through this capability, you can use a player of your own or a third party to take over web page media playback, improving playback experience. 10 11## Implementation Principle 12 13### Framework of Using the ArkWeb Kernel to Play Media 14 15When web media playback takeover is disabled, the playback architecture of the ArkWeb kernel is as follows: 16 17  18 19 > **NOTE** 20 > 21 > - In the preceding figure, step 1 indicates that the ArkWeb kernel creates a **WebMediaPlayer** to play media resources on web pages. 22 > - Step 2 indicates that the **WebMediaPlayer** uses the system decoder to render media data. 23 24When web media playback takeover is enabled, the playback architecture of the ArkWeb kernel is as follows: 25 26  27 28 > **NOTE** 29 > 30 > - In the preceding figure, step 1 indicates that the ArkWeb kernel creates a **WebMediaPlayer** to play media resources on web pages. 31 > - Step 2 indicates that **WebMediaPlayer** uses **NativeMediaPlayer** provided by the application to render media data. 32 33 34### Interactions Between the ArkWeb Kernel and Application 35 36  37 38 > **NOTE** 39 > 40 > - For details about step 1 in the preceding figure, see [Enabling Web Media Playback Takeover](#enabling-web-media-playback-takeover). 41 > - For details about step 2, see [Creating a Native Media Player](#creating-a-native-media-player). 42 > - For details about step 3, see [Drawing Native Media Player Components](#drawing-native-media-player-components). 43 > - For details about step 4, see [Executing Playback Control Commands Sent by ArkWeb Kernel to the Native Media Player](#executing-playback-control-commands-sent-by-arkweb-kernel-to-the-native-media-player). 44 > - For details about step 5, see [Notifying the Status Information of Native Media Player to the ArkWeb Kernel](#notifying-the-status-information-of-native-media-player-to-the-arkweb-kernel). 45 46## How to Develop 47 48### Enabling Web Media Playback Takeover 49 50Use the [enableNativeMediaPlayer](../reference/apis-arkweb/arkts-basic-components-web-attributes.md#enablenativemediaplayer12) API to enable the feature of taking over web page media playback. 51 52 ```ts 53 // xxx.ets 54 import { webview } from '@kit.ArkWeb'; 55 56 @Entry 57 @Component 58 struct WebComponent { 59 controller: webview.WebviewController = new webview.WebviewController(); 60 61 build() { 62 Column() { 63 Web({ src: 'www.example.com', controller: this.controller }) 64 .enableNativeMediaPlayer({ enable: true, shouldOverlay: false }) 65 } 66 } 67 } 68 ``` 69 70### Creating a Native Media Player 71 72After this feature is enabled, the ArkWeb kernel triggers the callback registered by [onCreateNativeMediaPlayer](../reference/apis-arkweb/arkts-apis-webview-WebviewController.md#oncreatenativemediaplayer12) when a media file needs to be played on the web page. 73 74The application needs to register a callback for creating a native media player by invoking **onCreateNativeMediaPlayer**. 75 76The callback function determines whether to create a native media player to take over the web page media resources based on the media information. 77 78 * If the application does not take over the web page media resource, **null** is returned in the callback. 79 * If the application takes over the web page media resource, a native media player instance is returned in the callback. 80 81The native media player needs to implement the [NativeMediaPlayerBridge](../reference/apis-arkweb/arkts-apis-webview-NativeMediaPlayerBridge.md) API so that the ArkWeb kernel can control the playback on the native media player. 82 83 ```ts 84 // xxx.ets 85 import { webview } from '@kit.ArkWeb'; 86 87 // Implement the webview.NativeMediaPlayerBridge API. 88 // The ArkWeb kernel calls the webview.NativeMediaPlayerBridge methods to control playback on NativeMediaPlayer. 89 class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge { 90 // ...Implement the APIs in NativeMediaPlayerBridge... 91 constructor(handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) {} 92 updateRect(x: number, y: number, width: number, height: number) {} 93 play() {} 94 pause() {} 95 seek(targetTime: number) {} 96 release() {} 97 setVolume(volume: number) {} 98 setMuted(muted: boolean) {} 99 setPlaybackRate(playbackRate: number) {} 100 enterFullscreen() {} 101 exitFullscreen() {} 102 } 103 104 @Entry 105 @Component 106 struct WebComponent { 107 controller: webview.WebviewController = new webview.WebviewController(); 108 109 build() { 110 Column() { 111 Web({ src: 'www.example.com', controller: this.controller }) 112 .enableNativeMediaPlayer({ enable: true, shouldOverlay: false }) 113 .onPageBegin((event) => { 114 this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => { 115 // Determine whether to take over the media. 116 if (!shouldHandle(mediaInfo)) { 117 // The native media player does not take over the media. 118 // Return null. The ArkWeb kernel will play the media with the web media player. 119 return null; 120 } 121 // Take over the web media. 122 // Return a native media player instance to the ArkWeb kernel. 123 let nativePlayer: webview.NativeMediaPlayerBridge = new NativeMediaPlayerImpl(handler, mediaInfo); 124 return nativePlayer; 125 }); 126 }) 127 } 128 } 129 } 130 131 // stub 132 function shouldHandle(mediaInfo: webview.MediaInfo) { 133 return true; 134 } 135 ``` 136 137### Drawing Native Media Player Components 138 139When an application takes over the media playback on web pages, it needs to draw the native media player component and video images on the surface provided by the ArkWeb kernel. The ArkWeb kernel combines the surface and the web page and displays them. 140 141This process is the same as that of [Rendering and Drawing XComponent+AVPlayer and Button Components at the Same Layer](web-same-layer.md) 142 1431. In the application startup phase, the application should save **UIContext** to use it in subsequent rendering and drawing processes at the same layer. 144 145 ```ts 146 // xxxAbility.ets 147 148 import { UIAbility } from '@kit.AbilityKit'; 149 import { window } from '@kit.ArkUI'; 150 151 export default class EntryAbility extends UIAbility { 152 onWindowStageCreate(windowStage: window.WindowStage): void { 153 windowStage.loadContent('pages/Index', (err, data) => { 154 if (err.code) { 155 return; 156 } 157 // Save UIContext, which will be used in subsequent rendering and drawing at the same layer. 158 AppStorage.setOrCreate<UIContext>("UIContext", windowStage.getMainWindowSync().getUIContext()); 159 }); 160 } 161 162 // ...Other APIs that need to be overridden... 163 } 164 ``` 165 1662. The application uses the surface created by the ArkWeb kernel for rendering and drawing at the same layer. 167 168 ```ts 169 // xxx.ets 170 import { webview } from '@kit.ArkWeb'; 171 import { BuilderNode, FrameNode, NodeController, NodeRenderType } from '@kit.ArkUI'; 172 173 interface ComponentParams {} 174 175 class MyNodeController extends NodeController { 176 private rootNode: BuilderNode<[ComponentParams]> | undefined; 177 178 constructor(surfaceId: string, renderType: NodeRenderType) { 179 super(); 180 181 // Obtain the saved UIContext. 182 let uiContext = AppStorage.get<UIContext>("UIContext"); 183 this.rootNode = new BuilderNode(uiContext as UIContext, { surfaceId: surfaceId, type: renderType }); 184 } 185 186 makeNode(uiContext: UIContext): FrameNode | null { 187 if (this.rootNode) { 188 return this.rootNode.getFrameNode() as FrameNode; 189 } 190 return null; 191 } 192 193 build() { 194 // Construct the native media player component. 195 } 196 } 197 198 @Entry 199 @Component 200 struct WebComponent { 201 node_controller?: MyNodeController; 202 controller: webview.WebviewController = new webview.WebviewController(); 203 @State show_native_media_player: boolean = false; 204 205 build() { 206 Column() { 207 Stack({ alignContent: Alignment.TopStart }) { 208 if (this.show_native_media_player) { 209 NodeContainer(this.node_controller) 210 .width(300) 211 .height(150) 212 .backgroundColor(Color.Transparent) 213 .border({ width: 2, color: Color.Orange }) 214 } 215 Web({ src: 'www.example.com', controller: this.controller }) 216 .enableNativeMediaPlayer({ enable: true, shouldOverlay: false }) 217 .onPageBegin((event) => { 218 this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => { 219 // Take over the web media. 220 221 // Use the surface provided by the rendering at the same layer to construct a native media player component. 222 this.node_controller = new MyNodeController(mediaInfo.surfaceInfo.id, NodeRenderType.RENDER_TYPE_TEXTURE); 223 this.node_controller.build(); 224 225 // Show the native media player component. 226 this.show_native_media_player = true; 227 228 // Return a native media player instance to the ArkWeb kernel. 229 return null; 230 }); 231 }) 232 } 233 } 234 } 235 } 236 ``` 237 238For details about how to dynamically create components and draw them on the surface, see [Rendering and Drawing XComponent+AVPlayer and Button Components at the Same Layer](web-same-layer.md). 239 240### Executing Playback Control Commands Sent by ArkWeb Kernel to the Native Media Player 241 242To facilitate the control over native media player by the ArkWeb kernel, the application needs to implement the [NativeMediaPlayerBridge](../reference/apis-arkweb/arkts-apis-webview-NativeMediaPlayerBridge.md) API on the native media player and operate the native media player based on the functionality of each API method. 243 244 ```ts 245 // xxx.ets 246 import { webview } from '@kit.ArkWeb'; 247 248 class ActualNativeMediaPlayerListener { 249 constructor(handler: webview.NativeMediaPlayerHandler) {} 250 } 251 252 class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge { 253 constructor(handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) { 254 // 1. Create a listener for the native media player. 255 let listener: ActualNativeMediaPlayerListener = new ActualNativeMediaPlayerListener(handler); 256 // 2. Create a native media player. 257 // 3. Listen for the native media player. 258 // ... 259 } 260 261 updateRect(x: number, y: number, width: number, height: number) { 262 // The position and size of the <video> tag are changed. 263 // Make changes based on the information change. 264 } 265 266 play() { 267 // Starts the native media player for playback. 268 } 269 270 pause() { 271 // Pause the playback. 272 } 273 274 seek(targetTime: number) { 275 // The native media player seeks to the target playback time. 276 } 277 278 release() { 279 // Destroy the native media player. 280 } 281 282 setVolume(volume: number) { 283 // The ArkWeb kernel adjusts the volume of the native media player. 284 // Set the volume of the native media player. 285 } 286 287 setMuted(muted: boolean) { 288 // Mute or unmute the native media player. 289 } 290 291 setPlaybackRate(playbackRate: number) { 292 // Set the playback rate of the native media player. 293 } 294 295 enterFullscreen() { 296 // Enter the full-screen mode. 297 } 298 299 exitFullscreen() { 300 // Exit the full-screen mode. 301 } 302 } 303 ``` 304 305### Notifying the Status Information of Native Media Player to the ArkWeb Kernel 306 307The ArkWeb kernel needs to update the status information (such as the video width and height, playback time, and cache time) of the native player to the web page. Therefore, the application needs to notify the ArkWeb kernel of the status information of the native player. 308 309In the [onCreateNativeMediaPlayer](../reference/apis-arkweb/arkts-apis-webview-WebviewController.md#oncreatenativemediaplayer12) API, the ArkWeb kernel passes a [NativeMediaPlayerHandler](../reference/apis-arkweb/arkts-apis-webview-NativeMediaPlayerHandler.md) object to the application. The application needs to use this object to notify the ArkWeb kernel of the latest status information of the native media player. 310 311 ```ts 312 // xxx.ets 313 import { webview } from '@kit.ArkWeb'; 314 315 class ActualNativeMediaPlayerListener { 316 handler: webview.NativeMediaPlayerHandler; 317 318 constructor(handler: webview.NativeMediaPlayerHandler) { 319 this.handler = handler; 320 } 321 322 onPlaying() { 323 // The native media player starts playback. 324 this.handler.handleStatusChanged(webview.PlaybackStatus.PLAYING); 325 } 326 onPaused() { 327 // The native media player pauses the playback. 328 this.handler.handleStatusChanged(webview.PlaybackStatus.PAUSED); 329 } 330 onSeeking() { 331 // The native media player starts to seek the target time point. 332 this.handler.handleSeeking(); 333 } 334 onSeekDone() { 335 // The target time point is sought. 336 this.handler.handleSeekFinished(); 337 } 338 onEnded() { 339 // The playback on the native media player is ended. 340 this.handler.handleEnded(); 341 } 342 onVolumeChanged() { 343 // Obtain the volume of the native media player. 344 let volume: number = getVolume(); 345 this.handler.handleVolumeChanged(volume); 346 } 347 onCurrentPlayingTimeUpdate() { 348 // Update the playback time. 349 let currentTime: number = getCurrentPlayingTime(); 350 // Convert the time unit to second. 351 let currentTimeInSeconds = convertToSeconds(currentTime); 352 this.handler.handleTimeUpdate(currentTimeInSeconds); 353 } 354 onBufferedChanged() { 355 // The buffer is changed. 356 // Obtain the cache duration of the native media player. 357 let bufferedEndTime: number = getCurrentBufferedTime(); 358 // Convert the time unit to second. 359 let bufferedEndTimeInSeconds = convertToSeconds(bufferedEndTime); 360 this.handler.handleBufferedEndTimeChanged(bufferedEndTimeInSeconds); 361 362 // Check the cache state. 363 // If the cache state changes, notify the ArkWeb engine of the cache state. 364 let lastReadyState: webview.ReadyState = getLastReadyState(); 365 let currentReadyState: webview.ReadyState = getCurrentReadyState(); 366 if (lastReadyState != currentReadyState) { 367 this.handler.handleReadyStateChanged(currentReadyState); 368 } 369 } 370 onEnterFullscreen() { 371 // The native media player enters the full-screen mode. 372 let isFullscreen: boolean = true; 373 this.handler.handleFullscreenChanged(isFullscreen); 374 } 375 onExitFullscreen() { 376 // The native media player exits the full-screen mode. 377 let isFullscreen: boolean = false; 378 this.handler.handleFullscreenChanged(isFullscreen); 379 } 380 onUpdateVideoSize(width: number, height: number) { 381 // Notify the ArkWeb kernel of the video width and height parsed by the native player. 382 this.handler.handleVideoSizeChanged(width, height); 383 } 384 onDurationChanged(duration: number) { 385 // Notify the ArkWeb kernel of the new media duration parsed by the native player. 386 this.handler.handleDurationChanged(duration); 387 } 388 onError(error: webview.MediaError, errorMessage: string) { 389 // Notify the ArkWeb kernel that an error occurs in the native player. 390 this.handler.handleError(error, errorMessage); 391 } 392 onNetworkStateChanged(state: webview.NetworkState) { 393 // Notify the ArkWeb kernel that the network state of the native player changes. 394 this.handler.handleNetworkStateChanged(state); 395 } 396 onPlaybackRateChanged(playbackRate: number) { 397 // Notify the ArkWeb kernel that the playback rate of the native player changes. 398 this.handler.handlePlaybackRateChanged(playbackRate); 399 } 400 onMutedChanged(muted: boolean) { 401 // Notify the ArkWeb kernel that the native player is muted. 402 this.handler.handleMutedChanged(muted); 403 } 404 405 // ...Listen for other states of the native player. 406 } 407 @Entry 408 @Component 409 struct WebComponent { 410 controller: webview.WebviewController = new webview.WebviewController(); 411 @State show_native_media_player: boolean = false; 412 413 build() { 414 Column() { 415 Web({ src: 'www.example.com', controller: this.controller }) 416 .enableNativeMediaPlayer({enable: true, shouldOverlay: false}) 417 .onPageBegin((event) => { 418 this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => { 419 // Take over the web media. 420 421 // Create a native media player instance. 422 // let nativePlayer: NativeMediaPlayerImpl = new NativeMediaPlayerImpl(handler, mediaInfo); 423 424 // Create a listener for the native media player state. 425 let nativeMediaPlayerListener: ActualNativeMediaPlayerListener = new ActualNativeMediaPlayerListener(handler); 426 // Listen for the native media player state. 427 // nativePlayer.setListener(nativeMediaPlayerListener); 428 429 // Return the native media player instance to the ArkWeb kernel. 430 return null; 431 }); 432 }) 433 } 434 } 435 } 436 437 // stub 438 function getVolume() { 439 return 1; 440 } 441 function getCurrentPlayingTime() { 442 return 1; 443 } 444 function getCurrentBufferedTime() { 445 return 1; 446 } 447 function convertToSeconds(input: number) { 448 return input; 449 } 450 function getLastReadyState() { 451 return webview.ReadyState.HAVE_NOTHING; 452 } 453 function getCurrentReadyState() { 454 return webview.ReadyState.HAVE_NOTHING; 455 } 456 ``` 457 458 459## Sample Code 460 461- Add the following permission to the **module.json5** file: 462 463 ```ts 464 "ohos.permission.INTERNET" 465 ``` 466 467- Save **UIContext** during application startup. 468 469 ```ts 470 // xxxAbility.ets 471 472 import { UIAbility } from '@kit.AbilityKit'; 473 import { window } from '@kit.ArkUI'; 474 475 export default class EntryAbility extends UIAbility { 476 onWindowStageCreate(windowStage: window.WindowStage): void { 477 windowStage.loadContent('pages/Index', (err, data) => { 478 if (err.code) { 479 return; 480 } 481 // Save UIContext, which will be used in subsequent rendering and drawing at the same layer. 482 AppStorage.setOrCreate<UIContext>("UIContext", windowStage.getMainWindowSync().getUIContext()); 483 }); 484 } 485 486 // ...Other APIs that need to be overridden... 487 } 488 ``` 489 490- Example of web media playback takeover: 491 492 ```ts 493 // Index.ets 494 import { webview } from '@kit.ArkWeb'; 495 import { BuilderNode, FrameNode, NodeController, NodeRenderType } from '@kit.ArkUI'; 496 import { AVPlayerDemo, AVPlayerListener } from './PlayerDemo'; 497 498 // Implement the webview.NativeMediaPlayerBridge API. 499 // The ArkWeb kernel calls the webview.NativeMediaPlayerBridge methods to control playback on NativeMediaPlayer. 500 class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge { 501 private surfaceId: string; 502 mediaSource: string; 503 private mediaHandler: webview.NativeMediaPlayerHandler; 504 nativePlayerInfo: NativePlayerInfo; 505 nativePlayer: AVPlayerDemo; 506 httpHeaders: Record<string, string>; 507 uiContext?: UIContext; 508 509 constructor(nativePlayerInfo: NativePlayerInfo, handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo, uiContext: UIContext) { 510 this.uiContext = uiContext; 511 console.log(`NativeMediaPlayerImpl.constructor, surface_id[${mediaInfo.surfaceInfo.id}]`); 512 this.nativePlayerInfo = nativePlayerInfo; 513 this.mediaHandler = handler; 514 this.surfaceId = mediaInfo.surfaceInfo.id; 515 this.mediaSource = mediaInfo.mediaSrcList.find((item)=>{item.source.indexOf('.mp4') > 0})?.source 516 || mediaInfo.mediaSrcList[0].source; 517 this.httpHeaders = mediaInfo.headers; 518 this.nativePlayer = new AVPlayerDemo(); 519 520 // Use the rendering function at the same layer to draw the video and its playback control components to the web page. 521 this.nativePlayerInfo.node_controller = new MyNodeController( 522 this.nativePlayerInfo, this.surfaceId, this.mediaHandler, this, NodeRenderType.RENDER_TYPE_TEXTURE); 523 this.nativePlayerInfo.node_controller.build(); 524 this.nativePlayerInfo.show_native_media_player = true; 525 526 console.log(`NativeMediaPlayerImpl.mediaSource: ${this.mediaSource}, headers: ${JSON.stringify(this.httpHeaders)}`); 527 } 528 529 updateRect(x: number, y: number, width: number, height: number): void { 530 let width_in_vp = this.uiContext!.px2vp(width); 531 let height_in_vp = this.uiContext!.px2vp(height); 532 console.log(`updateRect(${x}, ${y}, ${width}, ${height}), vp:{${width_in_vp}, ${height_in_vp}}`); 533 534 this.nativePlayerInfo.updateNativePlayerRect(x, y, width, height); 535 } 536 537 play() { 538 console.log('NativeMediaPlayerImpl.play'); 539 this.nativePlayer.play(); 540 } 541 pause() { 542 console.log('NativeMediaPlayerImpl.pause'); 543 this.nativePlayer.pause(); 544 } 545 seek(targetTime: number) { 546 console.log(`NativeMediaPlayerImpl.seek(${targetTime})`); 547 this.nativePlayer.seek(targetTime); 548 } 549 setVolume(volume: number) { 550 console.log(`NativeMediaPlayerImpl.setVolume(${volume})`); 551 this.nativePlayer.setVolume(volume); 552 } 553 setMuted(muted: boolean) { 554 console.log(`NativeMediaPlayerImpl.setMuted(${muted})`); 555 } 556 setPlaybackRate(playbackRate: number) { 557 console.log(`NativeMediaPlayerImpl.setPlaybackRate(${playbackRate})`); 558 this.nativePlayer.setPlaybackRate(playbackRate); 559 } 560 release() { 561 console.log('NativeMediaPlayerImpl.release'); 562 this.nativePlayer?.release(); 563 this.nativePlayerInfo.show_native_media_player = false; 564 this.nativePlayerInfo.node_width = 300; 565 this.nativePlayerInfo.node_height = 150; 566 this.nativePlayerInfo.destroyed(); 567 } 568 enterFullscreen() { 569 console.log('NativeMediaPlayerImpl.enterFullscreen'); 570 } 571 exitFullscreen() { 572 console.log('NativeMediaPlayerImpl.exitFullscreen'); 573 } 574 } 575 576 // Listen for the NativeMediaPlayer status and report the status to the ArkWeb kernel using webview.NativeMediaPlayerHandler. 577 class AVPlayerListenerImpl implements AVPlayerListener { 578 handler: webview.NativeMediaPlayerHandler; 579 component: NativePlayerComponent; 580 581 constructor(handler: webview.NativeMediaPlayerHandler, component: NativePlayerComponent) { 582 this.handler = handler; 583 this.component = component; 584 } 585 onPlaying() { 586 console.log('AVPlayerListenerImpl.onPlaying'); 587 this.handler.handleStatusChanged(webview.PlaybackStatus.PLAYING); 588 } 589 onPaused() { 590 console.log('AVPlayerListenerImpl.onPaused'); 591 this.handler.handleStatusChanged(webview.PlaybackStatus.PAUSED); 592 } 593 onDurationChanged(duration: number) { 594 console.log(`AVPlayerListenerImpl.onDurationChanged(${duration})`); 595 this.handler.handleDurationChanged(duration); 596 } 597 onBufferedTimeChanged(buffered: number) { 598 console.log(`AVPlayerListenerImpl.onBufferedTimeChanged(${buffered})`); 599 this.handler.handleBufferedEndTimeChanged(buffered); 600 } 601 onTimeUpdate(time: number) { 602 this.handler.handleTimeUpdate(time); 603 } 604 onEnded() { 605 console.log('AVPlayerListenerImpl.onEnded'); 606 this.handler.handleEnded(); 607 } 608 onError() { 609 console.log('AVPlayerListenerImpl.onError'); 610 this.component.has_error = true; 611 setTimeout(()=>{ 612 this.handler.handleError(1, "Oops!"); 613 }, 200); 614 } 615 onVideoSizeChanged(width: number, height: number) { 616 console.log(`AVPlayerListenerImpl.onVideoSizeChanged(${width}, ${height})`); 617 this.handler.handleVideoSizeChanged(width, height); 618 this.component.onSizeChanged(width, height); 619 } 620 onDestroyed(): void { 621 console.log('AVPlayerListenerImpl.onDestroyed'); 622 } 623 } 624 625 interface ComponentParams { 626 text: string; 627 text2: string; 628 playerInfo: NativePlayerInfo; 629 handler: webview.NativeMediaPlayerHandler; 630 player: NativeMediaPlayerImpl; 631 } 632 633 // Define the player components. 634 @Component 635 struct NativePlayerComponent { 636 params?: ComponentParams; 637 @State bgColor: Color = Color.Red; 638 mXComponentController: XComponentController = new XComponentController(); 639 640 videoController: VideoController = new VideoController(); 641 offset_x: number = 0; 642 offset_y: number = 0; 643 @State video_width_percent: number = 100; 644 @State video_height_percent: number = 100; 645 view_width: number = 0; 646 view_height: number = 0; 647 video_width: number = 0; 648 video_height: number = 0; 649 650 fullscreen: boolean = false; 651 @State has_error: boolean = false; 652 653 onSizeChanged(width: number, height: number) { 654 this.video_width = width; 655 this.video_height = height; 656 let scale: number = this.view_width / width; 657 let scaled_video_height: number = scale * height; 658 this.video_height_percent = scaled_video_height / this.view_height * 100; 659 console.log(`NativePlayerComponent.onSizeChanged(${width},${height}), video_height_percent[${this.video_height_percent }]`); 660 } 661 662 build() { 663 Column() { 664 Stack() { 665 XComponent({ id: 'video_player_id', type: XComponentType.SURFACE, controller: this.mXComponentController }) 666 .width(this.video_width_percent + '%') 667 .height(this.video_height_percent + '%') 668 .onLoad(()=>{ 669 if (!this.params) { 670 console.log('this.params is null'); 671 return; 672 } 673 console.log('NativePlayerComponent.onLoad, params[' + this.params 674 + '], text[' + this.params.text + '], text2[' + this.params.text2 675 + '], web_tab[' + this.params.playerInfo + '], handler[' + this.params.handler + ']'); 676 this.params.player.nativePlayer.setSurfaceID(this.mXComponentController.getXComponentSurfaceId()); 677 678 this.params.player.nativePlayer.avPlayerLiveDemo({ 679 url: this.params.player.mediaSource, 680 listener: new AVPlayerListenerImpl(this.params.handler, this), 681 httpHeaders: this.params.player.httpHeaders, 682 }); 683 }) 684 Column() { 685 Row() { 686 Button(this.params?.text) 687 .height(50) 688 .border({ width: 2, color: Color.Red }) 689 .backgroundColor(this.bgColor) 690 .onClick(()=>{ 691 console.log(`NativePlayerComponent.Button[${this.params?.text}] is clicked`); 692 this.params?.player.nativePlayer?.play(); 693 }) 694 .onTouch((event: TouchEvent) => { 695 event.stopPropagation(); 696 }) 697 Button(this.params?.text2) 698 .height(50) 699 .border({ width: 2, color: Color.Red }) 700 .onClick(()=>{ 701 console.log(`NativePlayerComponent.Button[${this.params?.text2}] is clicked`); 702 this.params?.player.nativePlayer?.pause(); 703 }) 704 .onTouch((event: TouchEvent) => { 705 event.stopPropagation(); 706 }) 707 } 708 .width('100%') 709 .justifyContent(FlexAlign.SpaceEvenly) 710 } 711 if (this.has_error) { 712 Column() { 713 Text('Error') 714 .fontSize(30) 715 } 716 .backgroundColor('#eb5555') 717 .width('100%') 718 .height('100%') 719 .justifyContent(FlexAlign.Center) 720 } 721 } 722 } 723 .width('100%') 724 .height('100%') 725 .onAreaChange((oldValue: Area, newValue: Area) => { 726 console.log(`NativePlayerComponent.onAreaChange(${JSON.stringify(oldValue)}, ${JSON.stringify(newValue)})`); 727 this.view_width = new Number(newValue.width).valueOf(); 728 this.view_height = new Number(newValue.height).valueOf(); 729 this.onSizeChanged(this.video_width, this.video_height); 730 }) 731 } 732 } 733 734 @Builder 735 function NativePlayerComponentBuilder(params: ComponentParams) { 736 NativePlayerComponent({ params: params }) 737 .backgroundColor(Color.Green) 738 .border({ width: 1, color: Color.Brown }) 739 .width('100%') 740 .height('100%') 741 } 742 743 // Use NodeController to dynamically create a custom player component and draw the component content on the surface. 744 class MyNodeController extends NodeController { 745 private rootNode: BuilderNode<[ComponentParams]> | undefined; 746 playerInfo: NativePlayerInfo; 747 listener: webview.NativeMediaPlayerHandler; 748 player: NativeMediaPlayerImpl; 749 750 constructor(playerInfo: NativePlayerInfo, 751 surfaceId: string, 752 listener: webview.NativeMediaPlayerHandler, 753 player: NativeMediaPlayerImpl, 754 renderType: NodeRenderType) { 755 super(); 756 this.playerInfo = playerInfo; 757 this.listener = listener; 758 this.player = player; 759 let uiContext = AppStorage.get<UIContext>("UIContext"); 760 this.rootNode = new BuilderNode(uiContext as UIContext, { surfaceId: surfaceId, type: renderType }); 761 console.log(`MyNodeController, rootNode[${this.rootNode}], playerInfo[${playerInfo}], listener[${listener}], surfaceId[${surfaceId}]`); 762 } 763 764 makeNode(): FrameNode | null { 765 if (this.rootNode) { 766 return this.rootNode.getFrameNode() as FrameNode; 767 } 768 return null; 769 } 770 771 build() { 772 let params: ComponentParams = { 773 "text": "play", 774 "text2": "pause", 775 playerInfo: this.playerInfo, 776 handler: this.listener, 777 player: this.player 778 }; 779 if (this.rootNode) { 780 this.rootNode.build(wrapBuilder(NativePlayerComponentBuilder), params); 781 } 782 } 783 784 postTouchEvent(event: TouchEvent) { 785 return this.rootNode?.postTouchEvent(event); 786 } 787 } 788 789 class Rect { 790 x: number = 0; 791 y: number = 0; 792 width: number = 0; 793 height: number = 0; 794 795 static toNodeRect(rectInPx: webview.RectEvent, uiContext: UIContext) : Rect { 796 let rect = new Rect(); 797 rect.x = uiContext.px2vp(rectInPx.x); 798 rect.y = uiContext.px2vp(rectInPx.x); 799 rect.width = uiContext.px2vp(rectInPx.width); 800 rect.height = uiContext.px2vp(rectInPx.height); 801 return rect; 802 } 803 } 804 805 @Observed 806 class NativePlayerInfo { 807 public web: WebComponent; 808 public embed_id: string; 809 public player: webview.NativeMediaPlayerBridge; 810 public node_controller?: MyNodeController; 811 public show_native_media_player: boolean = false; 812 public node_pos_x: number; 813 public node_pos_y: number; 814 public node_width: number; 815 public node_height: number; 816 817 playerComponent?: NativeMediaPlayerComponent; 818 819 constructor(web: WebComponent, handler: webview.NativeMediaPlayerHandler, videoInfo: webview.MediaInfo, uiContext: UIContext) { 820 this.web = web; 821 this.embed_id = videoInfo.embedID; 822 823 let node_rect = Rect.toNodeRect(videoInfo.surfaceInfo.rect, uiContext); 824 this.node_pos_x = node_rect.x; 825 this.node_pos_y = node_rect.y; 826 this.node_width = node_rect.width; 827 this.node_height = node_rect.height; 828 829 this.player = new NativeMediaPlayerImpl(this, handler, videoInfo, uiContext); 830 } 831 832 updateNativePlayerRect(x: number, y: number, width: number, height: number) { 833 this.playerComponent?.updateNativePlayerRect(x, y, width, height); 834 } 835 836 destroyed() { 837 let info_list = this.web.native_player_info_list; 838 console.log(`NativePlayerInfo[${this.embed_id}] destroyed, list.size[${info_list.length}]`); 839 this.web.native_player_info_list = info_list.filter((item) => item.embed_id != this.embed_id); 840 console.log(`NativePlayerInfo after destroyed, new_list.size[${this.web.native_player_info_list.length}]`); 841 } 842 } 843 844 @Component 845 struct NativeMediaPlayerComponent { 846 @ObjectLink playerInfo: NativePlayerInfo; 847 848 aboutToAppear() { 849 this.playerInfo.playerComponent = this; 850 } 851 852 build() { 853 NodeContainer(this.playerInfo.node_controller) 854 .width(this.playerInfo.node_width) 855 .height(this.playerInfo.node_height) 856 .offset({x: this.playerInfo.node_pos_x, y: this.playerInfo.node_pos_y}) 857 .backgroundColor(Color.Transparent) 858 .border({ width: 2, color: Color.Orange }) 859 .onAreaChange((oldValue, newValue) => { 860 console.log(`NodeContainer[${this.playerInfo.embed_id}].onAreaChange([${oldValue.width} x ${oldValue.height}]->[${newValue.width} x ${newValue.height}]`); 861 }) 862 } 863 864 updateNativePlayerRect(x: number, y: number, width: number, height: number) { 865 let node_rect = Rect.toNodeRect({x, y, width, height}, this.getUIContext()); 866 this.playerInfo.node_pos_x = node_rect.x; 867 this.playerInfo.node_pos_y = node_rect.y; 868 this.playerInfo.node_width = node_rect.width; 869 this.playerInfo.node_height = node_rect.height; 870 } 871 } 872 873 @Entry 874 @Component 875 struct WebComponent { 876 controller: WebviewController = new webview.WebviewController(); 877 page_url: Resource = $rawfile('main.html'); 878 879 @State native_player_info_list: NativePlayerInfo[] = []; 880 881 area?: Area; 882 883 build() { 884 Column() { 885 Stack({alignContent: Alignment.TopStart}) { 886 ForEach(this.native_player_info_list, (item: NativePlayerInfo) => { 887 if (item.show_native_media_player) { 888 NativeMediaPlayerComponent({ playerInfo: item }) 889 } 890 }, (item: NativePlayerInfo) => { 891 return item.embed_id; 892 }) 893 Web({ src: this.page_url, controller: this.controller }) 894 .enableNativeMediaPlayer({ enable: true, shouldOverlay: true }) 895 .onPageBegin(() => { 896 this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => { 897 console.log('onCreateNativeMediaPlayer(' + JSON.stringify(mediaInfo) + ')'); 898 let nativePlayerInfo = new NativePlayerInfo(this, handler, mediaInfo, this.getUIContext()); 899 this.native_player_info_list.push(nativePlayerInfo); 900 return nativePlayerInfo.player; 901 }); 902 }) 903 .onNativeEmbedGestureEvent((event)=>{ 904 if (!event.touchEvent || !event.embedId) { 905 event.result?.setGestureEventResult(false); 906 return; 907 } 908 console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}]`); 909 let native_player_info = this.getNativePlayerInfoByEmbedId(event.embedId); 910 if (!native_player_info) { 911 console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}], no native_player_info`); 912 event.result?.setGestureEventResult(false); 913 return; 914 } 915 if (!native_player_info.node_controller) { 916 console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}], no node_controller`); 917 event.result?.setGestureEventResult(false); 918 return; 919 } 920 let ret = native_player_info.node_controller.postTouchEvent(event.touchEvent); 921 console.log(`WebComponent.postTouchEvent, ret[${ret}], touchEvent[${JSON.stringify(event.touchEvent)}]`); 922 event.result?.setGestureEventResult(ret); 923 }) 924 .width('100%') 925 .height('100%') 926 .onAreaChange((oldValue: Area, newValue: Area) => { 927 oldValue; 928 this.area = newValue; 929 }) 930 } 931 } 932 } 933 934 getNativePlayerInfoByEmbedId(embedId: string) : NativePlayerInfo | undefined { 935 return this.native_player_info_list.find((item)=> item.embed_id == embedId); 936 } 937 } 938 ``` 939 940- Example of using video playback on the application side: 941 942 ```ts 943 import { media } from '@kit.MediaKit'; 944 import { BusinessError } from '@kit.BasicServicesKit'; 945 946 export interface AVPlayerListener { 947 onPlaying() : void; 948 onPaused() : void; 949 onDurationChanged(duration: number) : void; 950 onBufferedTimeChanged(buffered: number) : void; 951 onTimeUpdate(time: number) : void; 952 onEnded() : void; 953 onError() : void; 954 onVideoSizeChanged(width: number, height: number): void; 955 onDestroyed(): void; 956 } 957 958 interface PlayerParam { 959 url: string; 960 listener?: AVPlayerListener; 961 httpHeaders?: Record<string, string>; 962 } 963 964 interface PlayCommand { 965 func: Function; 966 name?: string; 967 } 968 969 interface CheckPlayCommandResult { 970 ignore: boolean; 971 index_to_remove: number; 972 } 973 974 export class AVPlayerDemo { 975 private surfaceID: string = ''; // The surfaceID parameter specifies the window used to display the video. Its value is obtained through the XComponent. 976 977 avPlayer?: media.AVPlayer; 978 prepared: boolean = false; 979 980 commands: PlayCommand[] = []; 981 982 setSurfaceID(surface_id: string) { 983 console.log(`AVPlayerDemo.setSurfaceID : ${surface_id}`); 984 this.surfaceID = surface_id; 985 } 986 // Set AVPlayer callback functions. 987 setAVPlayerCallback(avPlayer: media.AVPlayer, listener?: AVPlayerListener) { 988 // Callback function for the seek operation. 989 avPlayer.on('seekDone', (seekDoneTime: number) => { 990 console.info(`AVPlayer seek succeeded, seek time is ${seekDoneTime}`); 991 }); 992 // Callback function for errors. If an error occurs during the operation on the AVPlayer, reset() is called to reset the AVPlayer. 993 avPlayer.on('error', (err: BusinessError) => { 994 console.error(`Invoke avPlayer failed, code is ${err.code}, message is ${err.message}`); 995 listener?.onError(); 996 avPlayer.reset(); // Call reset() to reset the AVPlayer, which enters the idle state. 997 }); 998 // Callback for state changes. 999 avPlayer.on('stateChange', async (state: string, reason: media.StateChangeReason) => { 1000 switch (state) { 1001 case 'idle': // This state is reported upon a successful callback of reset(). 1002 console.info('AVPlayer state idle called.'); 1003 avPlayer.release(); // Call release() to release the instance. 1004 break; 1005 case 'initialized': // This state is reported when the AVPlayer sets the playback source. 1006 console.info('AVPlayer state initialized called.'); 1007 avPlayer.surfaceId = this.surfaceID; // Set the window to display the video. This setting is not required when a pure audio asset is to be played. 1008 avPlayer.prepare(); 1009 break; 1010 case 'prepared': // This state is reported upon a successful callback of prepare(). 1011 console.info('AVPlayer state prepared called.'); 1012 this.prepared = true; 1013 this.schedule(); 1014 break; 1015 case 'playing': // This state is reported upon a successful callback of play(). 1016 console.info('AVPlayer state playing called.'); 1017 listener?.onPlaying(); 1018 break; 1019 case 'paused': // This state is reported upon a successful callback of pause(). 1020 console.info('AVPlayer state paused called.'); 1021 listener?.onPaused(); 1022 break; 1023 case 'completed': // This state is reported upon the completion of the playback. 1024 console.info('AVPlayer state completed called.'); 1025 avPlayer.stop(); // Call stop() to stop the playback. 1026 break; 1027 case 'stopped': // This state is reported upon a successful callback of stop(). 1028 console.info('AVPlayer state stopped called.'); 1029 listener?.onEnded(); 1030 break; 1031 case 'released': 1032 this.prepared = false; 1033 listener?.onDestroyed(); 1034 console.info('AVPlayer state released called.'); 1035 break; 1036 default: 1037 console.info('AVPlayer state unknown called.'); 1038 break; 1039 } 1040 }); 1041 avPlayer.on('durationUpdate', (duration: number) => { 1042 console.info(`AVPlayer state durationUpdate success,new duration is :${duration}`); 1043 listener?.onDurationChanged(duration/1000); 1044 }); 1045 avPlayer.on('timeUpdate', (time:number) => { 1046 listener?.onTimeUpdate(time/1000); 1047 }); 1048 avPlayer.on('bufferingUpdate', (infoType: media.BufferingInfoType, value: number) => { 1049 console.info(`AVPlayer state bufferingUpdate success,and infoType value is:${infoType}, value is : ${value}`); 1050 if (infoType == media.BufferingInfoType.BUFFERING_PERCENT) { 1051 } 1052 listener?.onBufferedTimeChanged(value); 1053 }) 1054 avPlayer.on('videoSizeChange', (width: number, height: number) => { 1055 console.info(`AVPlayer state videoSizeChange success,and width is:${width}, height is : ${height}`); 1056 listener?.onVideoSizeChanged(width, height); 1057 }) 1058 } 1059 1060 // The following example uses a URL to specify the network address for playing live streams. 1061 async avPlayerLiveDemo(playerParam: PlayerParam) { 1062 // Create an AVPlayer instance. 1063 this.avPlayer = await media.createAVPlayer(); 1064 // Create a callback for state changes. 1065 this.setAVPlayerCallback(this.avPlayer, playerParam.listener); 1066 1067 let mediaSource: media.MediaSource = media.createMediaSourceWithUrl(playerParam.url, playerParam.httpHeaders); 1068 let strategy: media.PlaybackStrategy = { 1069 preferredWidth: 100, 1070 preferredHeight: 100, 1071 preferredBufferDuration: 100, 1072 preferredHdr: false 1073 }; 1074 this.avPlayer.setMediaSource(mediaSource, strategy); 1075 console.log(`AVPlayer url:[${playerParam.url}]`); 1076 } 1077 1078 schedule() { 1079 if (!this.avPlayer) { 1080 return; 1081 } 1082 if (!this.prepared) { 1083 return; 1084 } 1085 if (this.commands.length > 0) { 1086 let command = this.commands.shift(); 1087 if (command) { 1088 command.func(); 1089 } 1090 if (this.commands.length > 0) { 1091 setTimeout(() => { 1092 this.schedule(); 1093 }); 1094 } 1095 } 1096 } 1097 1098 private checkCommand(selfName: string, oppositeName: string) { 1099 let index_to_remove = -1; 1100 let ignore_this_action = false; 1101 let index = this.commands.length - 1; 1102 while (index >= 0) { 1103 if (this.commands[index].name == selfName) { 1104 ignore_this_action = true; 1105 break; 1106 } 1107 if (this.commands[index].name == oppositeName) { 1108 index_to_remove = index; 1109 break; 1110 } 1111 index--; 1112 } 1113 1114 let result : CheckPlayCommandResult = { 1115 ignore: ignore_this_action, 1116 index_to_remove: index_to_remove, 1117 }; 1118 return result; 1119 } 1120 1121 play() { 1122 let commandName = 'play'; 1123 let checkResult = this.checkCommand(commandName, 'pause'); 1124 if (checkResult.ignore) { 1125 console.log(`AVPlayer ${commandName} ignored.`); 1126 this.schedule(); 1127 return; 1128 } 1129 if (checkResult.index_to_remove >= 0) { 1130 let removedCommand = this.commands.splice(checkResult.index_to_remove, 1); 1131 console.log(`AVPlayer ${JSON.stringify(removedCommand)} removed.`); 1132 return; 1133 } 1134 this.commands.push({ func: ()=>{ 1135 console.info('AVPlayer.play()'); 1136 this.avPlayer?.play(); 1137 }, name: commandName}); 1138 this.schedule(); 1139 } 1140 pause() { 1141 let commandName = 'pause'; 1142 let checkResult = this.checkCommand(commandName, 'play'); 1143 console.log(`checkResult:${JSON.stringify(checkResult)}`); 1144 if (checkResult.ignore) { 1145 console.log(`AVPlayer ${commandName} ignored.`); 1146 this.schedule(); 1147 return; 1148 } 1149 if (checkResult.index_to_remove >= 0) { 1150 let removedCommand = this.commands.splice(checkResult.index_to_remove, 1); 1151 console.log(`AVPlayer ${JSON.stringify(removedCommand)} removed.`); 1152 return; 1153 } 1154 this.commands.push({ func: ()=>{ 1155 console.info('AVPlayer.pause()'); 1156 this.avPlayer?.pause(); 1157 }, name: commandName}); 1158 this.schedule(); 1159 } 1160 release() { 1161 this.commands.push({ func: ()=>{ 1162 console.info('AVPlayer.release()'); 1163 this.avPlayer?.release(); 1164 }}); 1165 this.schedule(); 1166 } 1167 seek(time: number) { 1168 this.commands.push({ func: ()=>{ 1169 console.info(`AVPlayer.seek(${time})`); 1170 this.avPlayer?.seek(time * 1000); 1171 }}); 1172 this.schedule(); 1173 } 1174 setVolume(volume: number) { 1175 this.commands.push({ func: ()=>{ 1176 console.info(`AVPlayer.setVolume(${volume})`); 1177 this.avPlayer?.setVolume(volume); 1178 }}); 1179 this.schedule(); 1180 } 1181 setPlaybackRate(playbackRate: number) { 1182 let speed = media.PlaybackSpeed.SPEED_FORWARD_1_00_X; 1183 let delta = 0.05; 1184 playbackRate += delta; 1185 if (playbackRate < 1) { 1186 speed = media.PlaybackSpeed.SPEED_FORWARD_0_75_X; 1187 } else if (playbackRate < 1.25) { 1188 speed = media.PlaybackSpeed.SPEED_FORWARD_1_00_X; 1189 } else if (playbackRate < 1.5) { 1190 speed = media.PlaybackSpeed.SPEED_FORWARD_1_25_X; 1191 } else if (playbackRate < 2) { 1192 speed = media.PlaybackSpeed.SPEED_FORWARD_1_75_X; 1193 } else { 1194 speed = media.PlaybackSpeed.SPEED_FORWARD_2_00_X; 1195 } 1196 this.commands.push({ func: ()=>{ 1197 console.info(`AVPlayer.setSpeed(${speed})`); 1198 this.avPlayer?.setSpeed(speed); 1199 }}); 1200 this.schedule(); 1201 } 1202 } 1203 ``` 1204 1205- Example of a frontend page: 1206 1207 ```html 1208 <html> 1209 <head> 1210 <title>Video Hosting Test html</title> 1211 <meta name="viewport" content="width=device-width"> 1212 </head> 1213 <body> 1214 <div> 1215 <!-- Replace the URL with the actual URL of the video source. --> 1216 <video src='https://xxx.xxx/demo.mp4' style='width: 100%'></video> 1217 </div> 1218 </body> 1219 </html> 1220 ``` 1221