• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1# Taking Over the Media Playback on Web Pages
2
3The **Web** component provides the capability for applications to take over media playback on web pages, which improves media playback qualities on the web page.
4
5## When to Use
6
7On web pages, media player one provides limited features, the resolution of media playback is low, and some videos cannot be played.
8
9Through this capability, you can use a player of your own or a third party to take over web page media playback, improving playback experience.
10
11## Implementation Principle
12
13### Framework of Using the ArkWeb Kernel to Play Media
14
15When web media playback takeover is disabled, the playback architecture of the ArkWeb kernel is as follows:
16
17  ![arkweb media pipeline](figures/arkweb_media_pipeline.png)
18
19  > **NOTE**
20  >
21  > - In the preceding figure, step 1 indicates that the ArkWeb kernel creates a **WebMediaPlayer** to play media resources on web pages.
22  > - Step 2 indicates that the **WebMediaPlayer** uses the system decoder to render media data.
23
24When web media playback takeover is enabled, the playback architecture of the ArkWeb kernel is as follows:
25
26  ![arkweb native media player](figures/arkweb_native_media_player.png)
27
28  > **NOTE**
29  >
30  > - In the preceding figure, step 1 indicates that the ArkWeb kernel creates a **WebMediaPlayer** to play media resources on web pages.
31  > - Step 2 indicates that **WebMediaPlayer** uses **NativeMediaPlayer** provided by the application to render media data.
32
33
34### Interactions Between the ArkWeb Kernel and Application
35
36  ![interactions between arkweb and native media player](figures/interactions_between_arkweb_and_native_media_player.png)
37
38  > **NOTE**
39  >
40  > - For details about step 1 in the preceding figure, see [Enabling Web Media Playback Takeover](#enabling-web-media-playback-takeover).
41  > - For details about step 2, see [Creating a Native Media Player](#creating-a-native-media-player).
42  > - For details about step 3, see [Drawing Native Media Player Components](#drawing-native-media-player-components).
43  > - For details about step 4, see [Executing Playback Control Commands Sent by ArkWeb Kernel to the Native Media Player](#executing-playback-control-commands-sent-by-arkweb-kernel-to-the-native-media-player).
44  > - For details about step 5, see [Notifying the Status Information of Native Media Player to the ArkWeb Kernel](#notifying-the-status-information-of-native-media-player-to-the-arkweb-kernel).
45
46## How to Develop
47
48### Enabling Web Media Playback Takeover
49
50Use the [enableNativeMediaPlayer](../reference/apis-arkweb/arkts-basic-components-web-attributes.md#enablenativemediaplayer12) API to enable the feature of taking over web page media playback.
51
52  ```ts
53  // xxx.ets
54  import { webview } from '@kit.ArkWeb';
55
56  @Entry
57  @Component
58  struct WebComponent {
59    controller: webview.WebviewController = new webview.WebviewController();
60
61    build() {
62      Column() {
63        Web({ src: 'www.example.com', controller: this.controller })
64          .enableNativeMediaPlayer({ enable: true, shouldOverlay: false })
65      }
66    }
67  }
68  ```
69
70### Creating a Native Media Player
71
72After this feature is enabled, the ArkWeb kernel triggers the callback registered by [onCreateNativeMediaPlayer](../reference/apis-arkweb/arkts-apis-webview-WebviewController.md#oncreatenativemediaplayer12) when a media file needs to be played on the web page.
73
74The application needs to register a callback for creating a native media player by invoking **onCreateNativeMediaPlayer**.
75
76The callback function determines whether to create a native media player to take over the web page media resources based on the media information.
77
78  * If the application does not take over the web page media resource, **null** is returned in the callback.
79  * If the application takes over the web page media resource, a native media player instance is returned in the callback.
80
81The native media player needs to implement the [NativeMediaPlayerBridge](../reference/apis-arkweb/arkts-apis-webview-NativeMediaPlayerBridge.md) API so that the ArkWeb kernel can control the playback on the native media player.
82
83  ```ts
84  // xxx.ets
85  import { webview } from '@kit.ArkWeb';
86
87  // Implement the webview.NativeMediaPlayerBridge API.
88  // The ArkWeb kernel calls the webview.NativeMediaPlayerBridge methods to control playback on NativeMediaPlayer.
89  class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge {
90    // ...Implement the APIs in NativeMediaPlayerBridge...
91    constructor(handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) {}
92    updateRect(x: number, y: number, width: number, height: number) {}
93    play() {}
94    pause() {}
95    seek(targetTime: number) {}
96    release() {}
97    setVolume(volume: number) {}
98    setMuted(muted: boolean) {}
99    setPlaybackRate(playbackRate: number) {}
100    enterFullscreen() {}
101    exitFullscreen() {}
102  }
103
104  @Entry
105  @Component
106  struct WebComponent {
107    controller: webview.WebviewController = new webview.WebviewController();
108
109    build() {
110      Column() {
111        Web({ src: 'www.example.com', controller: this.controller })
112          .enableNativeMediaPlayer({ enable: true, shouldOverlay: false })
113          .onPageBegin((event) => {
114            this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => {
115              // Determine whether to take over the media.
116              if (!shouldHandle(mediaInfo)) {
117                // The native media player does not take over the media.
118                // Return null. The ArkWeb kernel will play the media with the web media player.
119                return null;
120              }
121              // Take over the web media.
122              // Return a native media player instance to the ArkWeb kernel.
123              let nativePlayer: webview.NativeMediaPlayerBridge = new NativeMediaPlayerImpl(handler, mediaInfo);
124              return nativePlayer;
125            });
126          })
127      }
128    }
129  }
130
131  // stub
132  function shouldHandle(mediaInfo: webview.MediaInfo) {
133    return true;
134  }
135  ```
136
137### Drawing Native Media Player Components
138
139When an application takes over the media playback on web pages, it needs to draw the native media player component and video images on the surface provided by the ArkWeb kernel. The ArkWeb kernel combines the surface and the web page and displays them.
140
141This process is the same as that of [Rendering and Drawing XComponent+AVPlayer and Button Components at the Same Layer](web-same-layer.md)
142
1431. In the application startup phase, the application should save **UIContext** to use it in subsequent rendering and drawing processes at the same layer.
144
145   ```ts
146   // xxxAbility.ets
147
148   import { UIAbility } from '@kit.AbilityKit';
149   import { window } from '@kit.ArkUI';
150
151   export default class EntryAbility extends UIAbility {
152     onWindowStageCreate(windowStage: window.WindowStage): void {
153       windowStage.loadContent('pages/Index', (err, data) => {
154         if (err.code) {
155           return;
156         }
157         // Save UIContext, which will be used in subsequent rendering and drawing at the same layer.
158         AppStorage.setOrCreate<UIContext>("UIContext", windowStage.getMainWindowSync().getUIContext());
159       });
160     }
161
162     // ...Other APIs that need to be overridden...
163   }
164   ```
165
1662. The application uses the surface created by the ArkWeb kernel for rendering and drawing at the same layer.
167
168   ```ts
169   // xxx.ets
170   import { webview } from '@kit.ArkWeb';
171   import { BuilderNode, FrameNode, NodeController, NodeRenderType } from '@kit.ArkUI';
172
173   interface ComponentParams {}
174
175   class MyNodeController extends NodeController {
176     private rootNode: BuilderNode<[ComponentParams]> | undefined;
177
178     constructor(surfaceId: string, renderType: NodeRenderType) {
179       super();
180
181       // Obtain the saved UIContext.
182       let uiContext = AppStorage.get<UIContext>("UIContext");
183       this.rootNode = new BuilderNode(uiContext as UIContext, { surfaceId: surfaceId, type: renderType });
184     }
185
186     makeNode(uiContext: UIContext): FrameNode | null {
187       if (this.rootNode) {
188         return this.rootNode.getFrameNode() as FrameNode;
189       }
190       return null;
191     }
192
193     build() {
194       // Construct the native media player component.
195     }
196   }
197
198   @Entry
199   @Component
200   struct WebComponent {
201     node_controller?: MyNodeController;
202     controller: webview.WebviewController = new webview.WebviewController();
203     @State show_native_media_player: boolean = false;
204
205     build() {
206       Column() {
207         Stack({ alignContent: Alignment.TopStart }) {
208           if (this.show_native_media_player) {
209             NodeContainer(this.node_controller)
210               .width(300)
211               .height(150)
212               .backgroundColor(Color.Transparent)
213               .border({ width: 2, color: Color.Orange })
214           }
215           Web({ src: 'www.example.com', controller: this.controller })
216             .enableNativeMediaPlayer({ enable: true, shouldOverlay: false })
217             .onPageBegin((event) => {
218               this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo:    webview.MediaInfo) => {
219                 // Take over the web media.
220
221                 // Use the surface provided by the rendering at the same layer to construct a native media player component.
222                 this.node_controller = new MyNodeController(mediaInfo.surfaceInfo.id, NodeRenderType.RENDER_TYPE_TEXTURE);
223                 this.node_controller.build();
224
225                 // Show the native media player component.
226                 this.show_native_media_player = true;
227
228                 // Return a native media player instance to the ArkWeb kernel.
229                 return null;
230               });
231             })
232         }
233       }
234     }
235   }
236   ```
237
238For details about how to dynamically create components and draw them on the surface, see [Rendering and Drawing XComponent+AVPlayer and Button Components at the Same Layer](web-same-layer.md).
239
240### Executing Playback Control Commands Sent by ArkWeb Kernel to the Native Media Player
241
242To facilitate the control over native media player by the ArkWeb kernel, the application needs to implement the [NativeMediaPlayerBridge](../reference/apis-arkweb/arkts-apis-webview-NativeMediaPlayerBridge.md) API on the native media player and operate the native media player based on the functionality of each API method.
243
244  ```ts
245  // xxx.ets
246  import { webview } from '@kit.ArkWeb';
247
248  class ActualNativeMediaPlayerListener {
249    constructor(handler: webview.NativeMediaPlayerHandler) {}
250  }
251
252  class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge {
253    constructor(handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) {
254      // 1. Create a listener for the native media player.
255      let listener: ActualNativeMediaPlayerListener = new ActualNativeMediaPlayerListener(handler);
256      // 2. Create a native media player.
257      // 3. Listen for the native media player.
258      // ...
259    }
260
261    updateRect(x: number, y: number, width: number, height: number) {
262      // The position and size of the <video> tag are changed.
263      // Make changes based on the information change.
264    }
265
266    play() {
267      // Starts the native media player for playback.
268    }
269
270    pause() {
271      // Pause the playback.
272    }
273
274    seek(targetTime: number) {
275      // The native media player seeks to the target playback time.
276    }
277
278    release() {
279      // Destroy the native media player.
280    }
281
282    setVolume(volume: number) {
283      // The ArkWeb kernel adjusts the volume of the native media player.
284      // Set the volume of the native media player.
285    }
286
287    setMuted(muted: boolean) {
288      // Mute or unmute the native media player.
289    }
290
291    setPlaybackRate(playbackRate: number) {
292      // Set the playback rate of the native media player.
293    }
294
295    enterFullscreen() {
296      // Enter the full-screen mode.
297    }
298
299    exitFullscreen() {
300      // Exit the full-screen mode.
301    }
302  }
303  ```
304
305### Notifying the Status Information of Native Media Player to the ArkWeb Kernel
306
307The ArkWeb kernel needs to update the status information (such as the video width and height, playback time, and cache time) of the native player to the web page. Therefore, the application needs to notify the ArkWeb kernel of the status information of the native player.
308
309In the [onCreateNativeMediaPlayer](../reference/apis-arkweb/arkts-apis-webview-WebviewController.md#oncreatenativemediaplayer12) API, the ArkWeb kernel passes a [NativeMediaPlayerHandler](../reference/apis-arkweb/arkts-apis-webview-NativeMediaPlayerHandler.md) object to the application. The application needs to use this object to notify the ArkWeb kernel of the latest status information of the native media player.
310
311  ```ts
312  // xxx.ets
313  import { webview } from '@kit.ArkWeb';
314
315  class ActualNativeMediaPlayerListener {
316    handler: webview.NativeMediaPlayerHandler;
317
318    constructor(handler: webview.NativeMediaPlayerHandler) {
319      this.handler = handler;
320    }
321
322    onPlaying() {
323      // The native media player starts playback.
324      this.handler.handleStatusChanged(webview.PlaybackStatus.PLAYING);
325    }
326    onPaused() {
327      // The native media player pauses the playback.
328      this.handler.handleStatusChanged(webview.PlaybackStatus.PAUSED);
329    }
330    onSeeking() {
331      // The native media player starts to seek the target time point.
332      this.handler.handleSeeking();
333    }
334    onSeekDone() {
335      // The target time point is sought.
336      this.handler.handleSeekFinished();
337    }
338    onEnded() {
339      // The playback on the native media player is ended.
340      this.handler.handleEnded();
341    }
342    onVolumeChanged() {
343      // Obtain the volume of the native media player.
344      let volume: number = getVolume();
345      this.handler.handleVolumeChanged(volume);
346    }
347    onCurrentPlayingTimeUpdate() {
348      // Update the playback time.
349      let currentTime: number = getCurrentPlayingTime();
350      // Convert the time unit to second.
351      let currentTimeInSeconds = convertToSeconds(currentTime);
352      this.handler.handleTimeUpdate(currentTimeInSeconds);
353    }
354    onBufferedChanged() {
355      // The buffer is changed.
356      // Obtain the cache duration of the native media player.
357      let bufferedEndTime: number = getCurrentBufferedTime();
358      // Convert the time unit to second.
359      let bufferedEndTimeInSeconds = convertToSeconds(bufferedEndTime);
360      this.handler.handleBufferedEndTimeChanged(bufferedEndTimeInSeconds);
361
362      // Check the cache state.
363      // If the cache state changes, notify the ArkWeb engine of the cache state.
364      let lastReadyState: webview.ReadyState = getLastReadyState();
365      let currentReadyState: webview.ReadyState = getCurrentReadyState();
366      if (lastReadyState != currentReadyState) {
367        this.handler.handleReadyStateChanged(currentReadyState);
368      }
369    }
370    onEnterFullscreen() {
371      // The native media player enters the full-screen mode.
372      let isFullscreen: boolean = true;
373      this.handler.handleFullscreenChanged(isFullscreen);
374    }
375    onExitFullscreen() {
376      // The native media player exits the full-screen mode.
377      let isFullscreen: boolean = false;
378      this.handler.handleFullscreenChanged(isFullscreen);
379    }
380    onUpdateVideoSize(width: number, height: number) {
381      // Notify the ArkWeb kernel of the video width and height parsed by the native player.
382      this.handler.handleVideoSizeChanged(width, height);
383    }
384    onDurationChanged(duration: number) {
385      // Notify the ArkWeb kernel of the new media duration parsed by the native player.
386      this.handler.handleDurationChanged(duration);
387    }
388    onError(error: webview.MediaError, errorMessage: string) {
389      // Notify the ArkWeb kernel that an error occurs in the native player.
390      this.handler.handleError(error, errorMessage);
391    }
392    onNetworkStateChanged(state: webview.NetworkState) {
393      // Notify the ArkWeb kernel that the network state of the native player changes.
394      this.handler.handleNetworkStateChanged(state);
395    }
396    onPlaybackRateChanged(playbackRate: number) {
397      // Notify the ArkWeb kernel that the playback rate of the native player changes.
398      this.handler.handlePlaybackRateChanged(playbackRate);
399    }
400    onMutedChanged(muted: boolean) {
401      // Notify the ArkWeb kernel that the native player is muted.
402      this.handler.handleMutedChanged(muted);
403    }
404
405    // ...Listen for other states of the native player.
406  }
407  @Entry
408  @Component
409  struct WebComponent {
410    controller: webview.WebviewController = new webview.WebviewController();
411    @State show_native_media_player: boolean = false;
412
413    build() {
414      Column() {
415        Web({ src: 'www.example.com', controller: this.controller })
416          .enableNativeMediaPlayer({enable: true, shouldOverlay: false})
417          .onPageBegin((event) => {
418            this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => {
419              // Take over the web media.
420
421              // Create a native media player instance.
422              // let nativePlayer: NativeMediaPlayerImpl = new NativeMediaPlayerImpl(handler, mediaInfo);
423
424              // Create a listener for the native media player state.
425              let nativeMediaPlayerListener: ActualNativeMediaPlayerListener = new ActualNativeMediaPlayerListener(handler);
426              // Listen for the native media player state.
427              // nativePlayer.setListener(nativeMediaPlayerListener);
428
429              // Return the native media player instance to the ArkWeb kernel.
430              return null;
431            });
432          })
433      }
434    }
435  }
436
437  // stub
438  function getVolume() {
439    return 1;
440  }
441  function getCurrentPlayingTime() {
442    return 1;
443  }
444  function getCurrentBufferedTime() {
445    return 1;
446  }
447  function convertToSeconds(input: number) {
448    return input;
449  }
450  function getLastReadyState() {
451    return webview.ReadyState.HAVE_NOTHING;
452  }
453  function getCurrentReadyState() {
454    return webview.ReadyState.HAVE_NOTHING;
455  }
456  ```
457
458
459## Sample Code
460
461- Add the following permission to the **module.json5** file:
462
463  ```ts
464  "ohos.permission.INTERNET"
465  ```
466
467- Save **UIContext** during application startup.
468
469  ```ts
470  // xxxAbility.ets
471
472  import { UIAbility } from '@kit.AbilityKit';
473  import { window } from '@kit.ArkUI';
474
475  export default class EntryAbility extends UIAbility {
476    onWindowStageCreate(windowStage: window.WindowStage): void {
477      windowStage.loadContent('pages/Index', (err, data) => {
478        if (err.code) {
479          return;
480        }
481        // Save UIContext, which will be used in subsequent rendering and drawing at the same layer.
482        AppStorage.setOrCreate<UIContext>("UIContext", windowStage.getMainWindowSync().getUIContext());
483      });
484    }
485
486    // ...Other APIs that need to be overridden...
487  }
488  ```
489
490- Example of web media playback takeover:
491
492  ```ts
493  // Index.ets
494  import { webview } from '@kit.ArkWeb';
495  import { BuilderNode, FrameNode, NodeController, NodeRenderType } from '@kit.ArkUI';
496  import { AVPlayerDemo, AVPlayerListener } from './PlayerDemo';
497
498  // Implement the webview.NativeMediaPlayerBridge API.
499  // The ArkWeb kernel calls the webview.NativeMediaPlayerBridge methods to control playback on NativeMediaPlayer.
500  class NativeMediaPlayerImpl implements webview.NativeMediaPlayerBridge {
501    private surfaceId: string;
502    mediaSource: string;
503    private mediaHandler: webview.NativeMediaPlayerHandler;
504    nativePlayerInfo: NativePlayerInfo;
505    nativePlayer: AVPlayerDemo;
506    httpHeaders: Record<string, string>;
507    uiContext?: UIContext;
508
509    constructor(nativePlayerInfo: NativePlayerInfo, handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo, uiContext: UIContext) {
510      this.uiContext = uiContext;
511      console.log(`NativeMediaPlayerImpl.constructor, surface_id[${mediaInfo.surfaceInfo.id}]`);
512      this.nativePlayerInfo = nativePlayerInfo;
513      this.mediaHandler = handler;
514      this.surfaceId = mediaInfo.surfaceInfo.id;
515      this.mediaSource = mediaInfo.mediaSrcList.find((item)=>{item.source.indexOf('.mp4') > 0})?.source
516        || mediaInfo.mediaSrcList[0].source;
517      this.httpHeaders = mediaInfo.headers;
518      this.nativePlayer = new AVPlayerDemo();
519
520      // Use the rendering function at the same layer to draw the video and its playback control components to the web page.
521      this.nativePlayerInfo.node_controller = new MyNodeController(
522        this.nativePlayerInfo, this.surfaceId, this.mediaHandler, this, NodeRenderType.RENDER_TYPE_TEXTURE);
523      this.nativePlayerInfo.node_controller.build();
524      this.nativePlayerInfo.show_native_media_player = true;
525
526      console.log(`NativeMediaPlayerImpl.mediaSource: ${this.mediaSource}, headers: ${JSON.stringify(this.httpHeaders)}`);
527    }
528
529    updateRect(x: number, y: number, width: number, height: number): void {
530      let width_in_vp = this.uiContext!.px2vp(width);
531      let height_in_vp = this.uiContext!.px2vp(height);
532      console.log(`updateRect(${x}, ${y}, ${width}, ${height}), vp:{${width_in_vp}, ${height_in_vp}}`);
533
534      this.nativePlayerInfo.updateNativePlayerRect(x, y, width, height);
535    }
536
537    play() {
538      console.log('NativeMediaPlayerImpl.play');
539      this.nativePlayer.play();
540    }
541    pause() {
542      console.log('NativeMediaPlayerImpl.pause');
543      this.nativePlayer.pause();
544    }
545    seek(targetTime: number) {
546      console.log(`NativeMediaPlayerImpl.seek(${targetTime})`);
547      this.nativePlayer.seek(targetTime);
548    }
549    setVolume(volume: number) {
550      console.log(`NativeMediaPlayerImpl.setVolume(${volume})`);
551      this.nativePlayer.setVolume(volume);
552    }
553    setMuted(muted: boolean) {
554      console.log(`NativeMediaPlayerImpl.setMuted(${muted})`);
555    }
556    setPlaybackRate(playbackRate: number) {
557      console.log(`NativeMediaPlayerImpl.setPlaybackRate(${playbackRate})`);
558      this.nativePlayer.setPlaybackRate(playbackRate);
559    }
560    release() {
561      console.log('NativeMediaPlayerImpl.release');
562      this.nativePlayer?.release();
563      this.nativePlayerInfo.show_native_media_player = false;
564      this.nativePlayerInfo.node_width = 300;
565      this.nativePlayerInfo.node_height = 150;
566      this.nativePlayerInfo.destroyed();
567    }
568    enterFullscreen() {
569      console.log('NativeMediaPlayerImpl.enterFullscreen');
570    }
571    exitFullscreen() {
572      console.log('NativeMediaPlayerImpl.exitFullscreen');
573    }
574  }
575
576  // Listen for the NativeMediaPlayer status and report the status to the ArkWeb kernel using webview.NativeMediaPlayerHandler.
577  class AVPlayerListenerImpl implements AVPlayerListener {
578    handler: webview.NativeMediaPlayerHandler;
579    component: NativePlayerComponent;
580
581    constructor(handler: webview.NativeMediaPlayerHandler, component: NativePlayerComponent) {
582      this.handler = handler;
583      this.component = component;
584    }
585    onPlaying() {
586      console.log('AVPlayerListenerImpl.onPlaying');
587      this.handler.handleStatusChanged(webview.PlaybackStatus.PLAYING);
588    }
589    onPaused() {
590      console.log('AVPlayerListenerImpl.onPaused');
591      this.handler.handleStatusChanged(webview.PlaybackStatus.PAUSED);
592    }
593    onDurationChanged(duration: number) {
594      console.log(`AVPlayerListenerImpl.onDurationChanged(${duration})`);
595      this.handler.handleDurationChanged(duration);
596    }
597    onBufferedTimeChanged(buffered: number) {
598      console.log(`AVPlayerListenerImpl.onBufferedTimeChanged(${buffered})`);
599      this.handler.handleBufferedEndTimeChanged(buffered);
600    }
601    onTimeUpdate(time: number) {
602      this.handler.handleTimeUpdate(time);
603    }
604    onEnded() {
605      console.log('AVPlayerListenerImpl.onEnded');
606      this.handler.handleEnded();
607    }
608    onError() {
609      console.log('AVPlayerListenerImpl.onError');
610      this.component.has_error = true;
611      setTimeout(()=>{
612        this.handler.handleError(1, "Oops!");
613      }, 200);
614    }
615    onVideoSizeChanged(width: number, height: number) {
616      console.log(`AVPlayerListenerImpl.onVideoSizeChanged(${width}, ${height})`);
617      this.handler.handleVideoSizeChanged(width, height);
618      this.component.onSizeChanged(width, height);
619    }
620    onDestroyed(): void {
621      console.log('AVPlayerListenerImpl.onDestroyed');
622    }
623  }
624
625  interface ComponentParams {
626    text: string;
627    text2: string;
628    playerInfo: NativePlayerInfo;
629    handler: webview.NativeMediaPlayerHandler;
630    player: NativeMediaPlayerImpl;
631  }
632
633  // Define the player components.
634  @Component
635  struct NativePlayerComponent {
636    params?: ComponentParams;
637    @State bgColor: Color = Color.Red;
638    mXComponentController: XComponentController = new XComponentController();
639
640    videoController: VideoController = new VideoController();
641    offset_x: number = 0;
642    offset_y: number = 0;
643    @State video_width_percent: number = 100;
644    @State video_height_percent: number = 100;
645    view_width: number = 0;
646    view_height: number = 0;
647    video_width: number = 0;
648    video_height: number = 0;
649
650    fullscreen: boolean = false;
651    @State has_error: boolean = false;
652
653    onSizeChanged(width: number, height: number) {
654      this.video_width = width;
655      this.video_height = height;
656      let scale: number = this.view_width / width;
657      let scaled_video_height: number = scale * height;
658      this.video_height_percent = scaled_video_height / this.view_height * 100;
659      console.log(`NativePlayerComponent.onSizeChanged(${width},${height}), video_height_percent[${this.video_height_percent }]`);
660    }
661
662    build() {
663      Column() {
664        Stack() {
665          XComponent({ id: 'video_player_id', type: XComponentType.SURFACE, controller: this.mXComponentController })
666            .width(this.video_width_percent + '%')
667            .height(this.video_height_percent + '%')
668            .onLoad(()=>{
669              if (!this.params) {
670                console.log('this.params is null');
671                return;
672              }
673              console.log('NativePlayerComponent.onLoad, params[' + this.params
674                + '], text[' + this.params.text + '], text2[' + this.params.text2
675                + '], web_tab[' + this.params.playerInfo + '], handler[' + this.params.handler + ']');
676              this.params.player.nativePlayer.setSurfaceID(this.mXComponentController.getXComponentSurfaceId());
677
678              this.params.player.nativePlayer.avPlayerLiveDemo({
679                url: this.params.player.mediaSource,
680                listener: new AVPlayerListenerImpl(this.params.handler, this),
681                httpHeaders: this.params.player.httpHeaders,
682              });
683            })
684          Column() {
685            Row() {
686              Button(this.params?.text)
687                .height(50)
688                .border({ width: 2, color: Color.Red })
689                .backgroundColor(this.bgColor)
690                .onClick(()=>{
691                  console.log(`NativePlayerComponent.Button[${this.params?.text}] is clicked`);
692                  this.params?.player.nativePlayer?.play();
693                })
694                .onTouch((event: TouchEvent) => {
695                  event.stopPropagation();
696                })
697              Button(this.params?.text2)
698                .height(50)
699                .border({ width: 2, color: Color.Red })
700                .onClick(()=>{
701                  console.log(`NativePlayerComponent.Button[${this.params?.text2}] is clicked`);
702                  this.params?.player.nativePlayer?.pause();
703                })
704                .onTouch((event: TouchEvent) => {
705                  event.stopPropagation();
706                })
707            }
708            .width('100%')
709            .justifyContent(FlexAlign.SpaceEvenly)
710          }
711          if (this.has_error) {
712            Column() {
713              Text('Error')
714                .fontSize(30)
715            }
716            .backgroundColor('#eb5555')
717            .width('100%')
718            .height('100%')
719            .justifyContent(FlexAlign.Center)
720          }
721        }
722      }
723      .width('100%')
724      .height('100%')
725      .onAreaChange((oldValue: Area, newValue: Area) => {
726        console.log(`NativePlayerComponent.onAreaChange(${JSON.stringify(oldValue)}, ${JSON.stringify(newValue)})`);
727        this.view_width = new Number(newValue.width).valueOf();
728        this.view_height = new Number(newValue.height).valueOf();
729        this.onSizeChanged(this.video_width, this.video_height);
730      })
731    }
732  }
733
734  @Builder
735  function NativePlayerComponentBuilder(params: ComponentParams) {
736    NativePlayerComponent({ params: params })
737      .backgroundColor(Color.Green)
738      .border({ width: 1, color: Color.Brown })
739      .width('100%')
740      .height('100%')
741  }
742
743  // Use NodeController to dynamically create a custom player component and draw the component content on the surface.
744  class MyNodeController extends NodeController {
745    private rootNode: BuilderNode<[ComponentParams]> | undefined;
746    playerInfo: NativePlayerInfo;
747    listener: webview.NativeMediaPlayerHandler;
748    player: NativeMediaPlayerImpl;
749
750    constructor(playerInfo: NativePlayerInfo,
751                surfaceId: string,
752                listener: webview.NativeMediaPlayerHandler,
753                player: NativeMediaPlayerImpl,
754                renderType: NodeRenderType) {
755      super();
756      this.playerInfo = playerInfo;
757      this.listener = listener;
758      this.player = player;
759      let uiContext = AppStorage.get<UIContext>("UIContext");
760      this.rootNode = new BuilderNode(uiContext as UIContext, { surfaceId: surfaceId, type: renderType });
761      console.log(`MyNodeController, rootNode[${this.rootNode}], playerInfo[${playerInfo}], listener[${listener}], surfaceId[${surfaceId}]`);
762    }
763
764    makeNode(): FrameNode | null {
765      if (this.rootNode) {
766        return this.rootNode.getFrameNode() as FrameNode;
767      }
768      return null;
769    }
770
771    build() {
772      let params: ComponentParams = {
773        "text": "play",
774        "text2": "pause",
775        playerInfo: this.playerInfo,
776        handler: this.listener,
777        player: this.player
778      };
779      if (this.rootNode) {
780        this.rootNode.build(wrapBuilder(NativePlayerComponentBuilder), params);
781      }
782    }
783
784    postTouchEvent(event: TouchEvent) {
785      return this.rootNode?.postTouchEvent(event);
786    }
787  }
788
789  class Rect {
790    x: number = 0;
791    y: number = 0;
792    width: number = 0;
793    height: number = 0;
794
795    static toNodeRect(rectInPx: webview.RectEvent, uiContext: UIContext) : Rect {
796      let rect = new Rect();
797      rect.x = uiContext.px2vp(rectInPx.x);
798      rect.y = uiContext.px2vp(rectInPx.x);
799      rect.width = uiContext.px2vp(rectInPx.width);
800      rect.height = uiContext.px2vp(rectInPx.height);
801      return rect;
802    }
803  }
804
805  @Observed
806  class NativePlayerInfo {
807    public web: WebComponent;
808    public embed_id: string;
809    public player: webview.NativeMediaPlayerBridge;
810    public node_controller?: MyNodeController;
811    public show_native_media_player: boolean = false;
812    public node_pos_x: number;
813    public node_pos_y: number;
814    public node_width: number;
815    public node_height: number;
816
817    playerComponent?: NativeMediaPlayerComponent;
818
819    constructor(web: WebComponent, handler: webview.NativeMediaPlayerHandler, videoInfo: webview.MediaInfo, uiContext: UIContext) {
820      this.web = web;
821      this.embed_id = videoInfo.embedID;
822
823      let node_rect = Rect.toNodeRect(videoInfo.surfaceInfo.rect, uiContext);
824      this.node_pos_x = node_rect.x;
825      this.node_pos_y = node_rect.y;
826      this.node_width = node_rect.width;
827      this.node_height = node_rect.height;
828
829      this.player = new NativeMediaPlayerImpl(this, handler, videoInfo, uiContext);
830    }
831
832    updateNativePlayerRect(x: number, y: number, width: number, height: number) {
833      this.playerComponent?.updateNativePlayerRect(x, y, width, height);
834    }
835
836    destroyed() {
837      let info_list = this.web.native_player_info_list;
838      console.log(`NativePlayerInfo[${this.embed_id}] destroyed, list.size[${info_list.length}]`);
839      this.web.native_player_info_list = info_list.filter((item) => item.embed_id != this.embed_id);
840      console.log(`NativePlayerInfo after destroyed, new_list.size[${this.web.native_player_info_list.length}]`);
841    }
842  }
843
844  @Component
845  struct NativeMediaPlayerComponent {
846    @ObjectLink playerInfo: NativePlayerInfo;
847
848    aboutToAppear() {
849      this.playerInfo.playerComponent = this;
850    }
851
852    build() {
853      NodeContainer(this.playerInfo.node_controller)
854        .width(this.playerInfo.node_width)
855        .height(this.playerInfo.node_height)
856        .offset({x: this.playerInfo.node_pos_x, y: this.playerInfo.node_pos_y})
857        .backgroundColor(Color.Transparent)
858        .border({ width: 2, color: Color.Orange })
859        .onAreaChange((oldValue, newValue) => {
860          console.log(`NodeContainer[${this.playerInfo.embed_id}].onAreaChange([${oldValue.width} x ${oldValue.height}]->[${newValue.width} x ${newValue.height}]`);
861        })
862    }
863
864    updateNativePlayerRect(x: number, y: number, width: number, height: number) {
865      let node_rect = Rect.toNodeRect({x, y, width, height}, this.getUIContext());
866      this.playerInfo.node_pos_x = node_rect.x;
867      this.playerInfo.node_pos_y = node_rect.y;
868      this.playerInfo.node_width = node_rect.width;
869      this.playerInfo.node_height = node_rect.height;
870    }
871  }
872
873  @Entry
874  @Component
875  struct WebComponent {
876    controller: WebviewController = new webview.WebviewController();
877    page_url: Resource = $rawfile('main.html');
878
879    @State native_player_info_list: NativePlayerInfo[] = [];
880
881    area?: Area;
882
883    build() {
884      Column() {
885        Stack({alignContent: Alignment.TopStart}) {
886          ForEach(this.native_player_info_list, (item: NativePlayerInfo) => {
887            if (item.show_native_media_player) {
888              NativeMediaPlayerComponent({ playerInfo: item })
889            }
890          }, (item: NativePlayerInfo) => {
891            return item.embed_id;
892          })
893          Web({ src: this.page_url, controller: this.controller })
894            .enableNativeMediaPlayer({ enable: true, shouldOverlay: true })
895            .onPageBegin(() => {
896              this.controller.onCreateNativeMediaPlayer((handler: webview.NativeMediaPlayerHandler, mediaInfo: webview.MediaInfo) => {
897                console.log('onCreateNativeMediaPlayer(' + JSON.stringify(mediaInfo) + ')');
898                let nativePlayerInfo = new NativePlayerInfo(this, handler, mediaInfo, this.getUIContext());
899                this.native_player_info_list.push(nativePlayerInfo);
900                return nativePlayerInfo.player;
901              });
902            })
903            .onNativeEmbedGestureEvent((event)=>{
904              if (!event.touchEvent || !event.embedId) {
905                event.result?.setGestureEventResult(false);
906                return;
907              }
908              console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}]`);
909              let native_player_info = this.getNativePlayerInfoByEmbedId(event.embedId);
910              if (!native_player_info) {
911                console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}], no native_player_info`);
912                event.result?.setGestureEventResult(false);
913                return;
914              }
915              if (!native_player_info.node_controller) {
916                console.log(`WebComponent.onNativeEmbedGestureEvent, embedId[${event.embedId}], no node_controller`);
917                event.result?.setGestureEventResult(false);
918                return;
919              }
920              let ret = native_player_info.node_controller.postTouchEvent(event.touchEvent);
921              console.log(`WebComponent.postTouchEvent, ret[${ret}], touchEvent[${JSON.stringify(event.touchEvent)}]`);
922              event.result?.setGestureEventResult(ret);
923            })
924            .width('100%')
925            .height('100%')
926            .onAreaChange((oldValue: Area, newValue: Area) => {
927              oldValue;
928              this.area = newValue;
929            })
930        }
931      }
932    }
933
934    getNativePlayerInfoByEmbedId(embedId: string) : NativePlayerInfo | undefined {
935      return this.native_player_info_list.find((item)=> item.embed_id == embedId);
936    }
937  }
938  ```
939
940- Example of using video playback on the application side:
941
942  ```ts
943  import { media } from '@kit.MediaKit';
944  import { BusinessError } from '@kit.BasicServicesKit';
945
946  export interface AVPlayerListener {
947    onPlaying() : void;
948    onPaused() : void;
949    onDurationChanged(duration: number) : void;
950    onBufferedTimeChanged(buffered: number) : void;
951    onTimeUpdate(time: number) : void;
952    onEnded() : void;
953    onError() : void;
954    onVideoSizeChanged(width: number, height: number): void;
955    onDestroyed(): void;
956  }
957
958  interface PlayerParam {
959    url: string;
960    listener?: AVPlayerListener;
961    httpHeaders?: Record<string, string>;
962  }
963
964  interface PlayCommand {
965    func: Function;
966    name?: string;
967  }
968
969  interface CheckPlayCommandResult {
970    ignore: boolean;
971    index_to_remove: number;
972  }
973
974  export class AVPlayerDemo {
975    private surfaceID: string = ''; // The surfaceID parameter specifies the window used to display the video. Its value is obtained through the XComponent.
976
977    avPlayer?: media.AVPlayer;
978    prepared: boolean = false;
979
980    commands: PlayCommand[] = [];
981
982    setSurfaceID(surface_id: string) {
983      console.log(`AVPlayerDemo.setSurfaceID : ${surface_id}`);
984      this.surfaceID = surface_id;
985    }
986    // Set AVPlayer callback functions.
987    setAVPlayerCallback(avPlayer: media.AVPlayer, listener?: AVPlayerListener) {
988      // Callback function for the seek operation.
989      avPlayer.on('seekDone', (seekDoneTime: number) => {
990        console.info(`AVPlayer seek succeeded, seek time is ${seekDoneTime}`);
991      });
992      // Callback function for errors. If an error occurs during the operation on the AVPlayer, reset() is called to reset the AVPlayer.
993      avPlayer.on('error', (err: BusinessError) => {
994        console.error(`Invoke avPlayer failed, code is ${err.code}, message is ${err.message}`);
995        listener?.onError();
996        avPlayer.reset(); // Call reset() to reset the AVPlayer, which enters the idle state.
997      });
998      // Callback for state changes.
999      avPlayer.on('stateChange', async (state: string, reason: media.StateChangeReason) => {
1000        switch (state) {
1001          case 'idle': // This state is reported upon a successful callback of reset().
1002            console.info('AVPlayer state idle called.');
1003            avPlayer.release(); // Call release() to release the instance.
1004            break;
1005          case 'initialized': // This state is reported when the AVPlayer sets the playback source.
1006            console.info('AVPlayer state initialized called.');
1007            avPlayer.surfaceId = this.surfaceID; // Set the window to display the video. This setting is not required when a pure audio asset is to be played.
1008            avPlayer.prepare();
1009            break;
1010          case 'prepared': // This state is reported upon a successful callback of prepare().
1011            console.info('AVPlayer state prepared called.');
1012            this.prepared = true;
1013            this.schedule();
1014            break;
1015          case 'playing': // This state is reported upon a successful callback of play().
1016            console.info('AVPlayer state playing called.');
1017            listener?.onPlaying();
1018            break;
1019          case 'paused': // This state is reported upon a successful callback of pause().
1020            console.info('AVPlayer state paused called.');
1021            listener?.onPaused();
1022            break;
1023          case 'completed': // This state is reported upon the completion of the playback.
1024            console.info('AVPlayer state completed called.');
1025            avPlayer.stop(); // Call stop() to stop the playback.
1026            break;
1027          case 'stopped': // This state is reported upon a successful callback of stop().
1028            console.info('AVPlayer state stopped called.');
1029            listener?.onEnded();
1030            break;
1031          case 'released':
1032            this.prepared = false;
1033            listener?.onDestroyed();
1034            console.info('AVPlayer state released called.');
1035            break;
1036          default:
1037            console.info('AVPlayer state unknown called.');
1038            break;
1039        }
1040      });
1041      avPlayer.on('durationUpdate', (duration: number) => {
1042        console.info(`AVPlayer state durationUpdate success,new duration is :${duration}`);
1043        listener?.onDurationChanged(duration/1000);
1044      });
1045      avPlayer.on('timeUpdate', (time:number) => {
1046        listener?.onTimeUpdate(time/1000);
1047      });
1048      avPlayer.on('bufferingUpdate', (infoType: media.BufferingInfoType, value: number) => {
1049        console.info(`AVPlayer state bufferingUpdate success,and infoType value is:${infoType}, value is : ${value}`);
1050        if (infoType == media.BufferingInfoType.BUFFERING_PERCENT) {
1051        }
1052        listener?.onBufferedTimeChanged(value);
1053      })
1054      avPlayer.on('videoSizeChange', (width: number, height: number) => {
1055        console.info(`AVPlayer state videoSizeChange success,and width is:${width}, height is : ${height}`);
1056        listener?.onVideoSizeChanged(width, height);
1057      })
1058    }
1059
1060    // The following example uses a URL to specify the network address for playing live streams.
1061    async avPlayerLiveDemo(playerParam: PlayerParam) {
1062      // Create an AVPlayer instance.
1063      this.avPlayer = await media.createAVPlayer();
1064      // Create a callback for state changes.
1065      this.setAVPlayerCallback(this.avPlayer, playerParam.listener);
1066
1067      let mediaSource: media.MediaSource = media.createMediaSourceWithUrl(playerParam.url, playerParam.httpHeaders);
1068      let strategy: media.PlaybackStrategy = {
1069        preferredWidth: 100,
1070        preferredHeight: 100,
1071        preferredBufferDuration: 100,
1072        preferredHdr: false
1073      };
1074      this.avPlayer.setMediaSource(mediaSource, strategy);
1075      console.log(`AVPlayer url:[${playerParam.url}]`);
1076    }
1077
1078    schedule() {
1079      if (!this.avPlayer) {
1080        return;
1081      }
1082      if (!this.prepared) {
1083        return;
1084      }
1085      if (this.commands.length > 0) {
1086        let command = this.commands.shift();
1087        if (command) {
1088          command.func();
1089        }
1090        if (this.commands.length > 0) {
1091          setTimeout(() => {
1092            this.schedule();
1093          });
1094        }
1095      }
1096    }
1097
1098    private checkCommand(selfName: string, oppositeName: string) {
1099      let index_to_remove = -1;
1100      let ignore_this_action = false;
1101      let index = this.commands.length - 1;
1102      while (index >= 0) {
1103        if (this.commands[index].name == selfName) {
1104          ignore_this_action = true;
1105          break;
1106        }
1107        if (this.commands[index].name == oppositeName) {
1108          index_to_remove = index;
1109          break;
1110        }
1111        index--;
1112      }
1113
1114      let result : CheckPlayCommandResult = {
1115        ignore: ignore_this_action,
1116        index_to_remove: index_to_remove,
1117      };
1118      return result;
1119    }
1120
1121    play() {
1122      let commandName = 'play';
1123      let checkResult = this.checkCommand(commandName, 'pause');
1124      if (checkResult.ignore) {
1125        console.log(`AVPlayer ${commandName} ignored.`);
1126        this.schedule();
1127        return;
1128      }
1129      if (checkResult.index_to_remove >= 0) {
1130        let removedCommand = this.commands.splice(checkResult.index_to_remove, 1);
1131        console.log(`AVPlayer ${JSON.stringify(removedCommand)} removed.`);
1132        return;
1133      }
1134      this.commands.push({ func: ()=>{
1135        console.info('AVPlayer.play()');
1136        this.avPlayer?.play();
1137      }, name: commandName});
1138      this.schedule();
1139    }
1140    pause() {
1141      let commandName = 'pause';
1142      let checkResult = this.checkCommand(commandName, 'play');
1143      console.log(`checkResult:${JSON.stringify(checkResult)}`);
1144      if (checkResult.ignore) {
1145        console.log(`AVPlayer ${commandName} ignored.`);
1146        this.schedule();
1147        return;
1148      }
1149      if (checkResult.index_to_remove >= 0) {
1150        let removedCommand = this.commands.splice(checkResult.index_to_remove, 1);
1151        console.log(`AVPlayer ${JSON.stringify(removedCommand)} removed.`);
1152        return;
1153      }
1154      this.commands.push({ func: ()=>{
1155        console.info('AVPlayer.pause()');
1156        this.avPlayer?.pause();
1157      }, name: commandName});
1158      this.schedule();
1159    }
1160    release() {
1161      this.commands.push({ func: ()=>{
1162        console.info('AVPlayer.release()');
1163        this.avPlayer?.release();
1164      }});
1165      this.schedule();
1166    }
1167    seek(time: number) {
1168      this.commands.push({ func: ()=>{
1169        console.info(`AVPlayer.seek(${time})`);
1170        this.avPlayer?.seek(time * 1000);
1171      }});
1172      this.schedule();
1173    }
1174    setVolume(volume: number) {
1175      this.commands.push({ func: ()=>{
1176        console.info(`AVPlayer.setVolume(${volume})`);
1177        this.avPlayer?.setVolume(volume);
1178      }});
1179      this.schedule();
1180    }
1181    setPlaybackRate(playbackRate: number) {
1182      let speed = media.PlaybackSpeed.SPEED_FORWARD_1_00_X;
1183      let delta = 0.05;
1184      playbackRate += delta;
1185      if (playbackRate < 1) {
1186        speed = media.PlaybackSpeed.SPEED_FORWARD_0_75_X;
1187      } else if (playbackRate < 1.25) {
1188        speed = media.PlaybackSpeed.SPEED_FORWARD_1_00_X;
1189      } else if (playbackRate < 1.5) {
1190        speed = media.PlaybackSpeed.SPEED_FORWARD_1_25_X;
1191      } else if (playbackRate < 2) {
1192        speed = media.PlaybackSpeed.SPEED_FORWARD_1_75_X;
1193      } else {
1194        speed = media.PlaybackSpeed.SPEED_FORWARD_2_00_X;
1195      }
1196      this.commands.push({ func: ()=>{
1197        console.info(`AVPlayer.setSpeed(${speed})`);
1198        this.avPlayer?.setSpeed(speed);
1199      }});
1200      this.schedule();
1201    }
1202  }
1203  ```
1204
1205- Example of a frontend page:
1206
1207  ```html
1208  <html>
1209  <head>
1210      <title>Video Hosting Test html</title>
1211      <meta name="viewport" content="width=device-width">
1212  </head>
1213  <body>
1214  <div>
1215      <!-- Replace the URL with the actual URL of the video source. -->
1216      <video src='https://xxx.xxx/demo.mp4' style='width: 100%'></video>
1217  </div>
1218  </body>
1219  </html>
1220  ```
1221