Last updated: 2021-09-14 14:30:22

    Overview

    Publishing from camera refers to the process of collecting video and audio data from the mobile phone’s camera and mic, encoding the data, and publishing it to cloud-based live streaming platforms. Tencent Cloud’s LiteAVSDK provides the camera publishing capability via V2TXLivePusher.

    Notes

    About running projects on x86 emulators: The SDK uses a lot of audio and video APIs of the iOS system, most of which cannot be used on the x86 emulator built into macOS. Therefore, we recommend that you test your project on a real device.

    Sample Code

    Platform GitHub Address Key Class
    iOS GitHub CameraPushViewController.m
    Android GitHub CameraPushMainActivity.java
    Note:

    In addition to the above sample code, regarding frequently asked questions among developers, Tencent Cloud offers a straightforward API example project, which you can use to quickly learn how to use different APIs.

    Integration

    1. Download the SDK

    Download the SDK ZIP file and integrate the SDK into your application as instructed in the SDK integration document.

    2. Configure a license for the SDK

    Click Get License to obtain a trial license. You will get two strings: a license URL and a decryption key.
    Before you use LiteAVSDK features in your application, complete the following configurations (preferably in - [AppDelegate application:didFinishLaunchingWithOptions:]):

    @import TXLiteAVSDK_Professional;
    @implementation AppDelegate
    - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {
       NSString * const licenceURL = @"<The license URL obtained>";
       NSString * const licenceKey = @"<The key obtained>";
    
       //TXLiveBase can be found in the "TXLiveBase.h" header file
       [TXLiveBase setLicenceURL:licenceURL key:licenceKey]; 
       NSLog(@"SDK Version = %@", [TXLiveBase getSDKVersionStr]);
    }
    @end
    

    3. Initialize the V2TXLivePusher component

    Create a V2TXLivePusher object and specify V2TXLiveMode.

    V2TXLivePusher *pusher = [[V2TXLivePusher alloc] initWithLiveMode:V2TXLiveMode_RTMP]; // Set the live streaming protocol to RTMP 
    

    4. Enable camera preview

    Call setRenderView in V2TXLivePusher to configure a view object for displaying video images, and then call startCamera to enable camera preview for your mobile phone.

    // Create a view object and insert it into the UI
    UIView *_localView = [[UIView alloc] initWithFrame:self.view.bounds];
    [self.view insertSubview:_localView atIndex:0];
    _localView.center = self.view.center;
    // Enable preview for the local camera
    [_pusher setRenderView:_localView];
    [_pusher startCamera:YES];
    Note:

    To add animated effects to the view, modify its transform attribute rather than frame attribute.

      [UIView animateWithDuration:0.5 animations:^{
                _localView.transform = CGAffineTransformMakeScale(0.3, 0.3); // Shrink by 1/3
            }];
    

    5. Start and stop publishing streams

    After calling startCamera to enable camera preview, you can call the startPush API in V2TXLivePusher to start stream publishing.

    // Start publishing streams
    NSString* rtmpUrl = @"rtmp://test.com/live/xxxxxx"; // Enter your RTMP URL for stream publishing
    [_pusher startPush:rtmpUrl];

    Call stopPush in V2TXLivePusher to stop publishing streams.

    // Stop publishing streams
    [_pusher stopPush];
    
    Note:

    If you have enabled camera preview, please disable it when you stop publishing streams.

    6. Publish audio-only streams

    If your live streaming scenarios involve audio only, you can skip Step 4 or do not call startCamera before startPush.

    V2TXLivePusher *_pusher = [[V2TXLivePusher alloc] initWithLiveMode:V2TXLiveMode_RTMP]; 
    NSString* rtmpUrl = @"rtmp://test.com/live/xxxxxx";    
    [_pusher startMicrophone];
    [_pusher startPush:rtmpUrl];
    
    Note:

    If you publish audio-only streams but no streams can be pulled from an RTMP, FLV, or HLS playback URL, there is a problem with your line configuration, please submit a ticket for help.

    7. Set video quality

    Call setVideoQuality in V2TXLivePusher to set the quality of videos watched by audience. The encoding parameters set determine the quality of videos presented to audience. The local video watched by the host is the original HD version that has not been encoded or compressed, and is therefore not affected by the settings. For details, please see Setting Video Quality.

    8. Set the beauty filter style and skin brightening and rosy skin effects

    Call getBeautyManager in V2TXLivePusher to get a TXBeautyManager instance to set beauty filters.

    Beauty filter style

    The SDK has three built-in beauty filter algorithms, each corresponding to a beauty filter style. Choose one that best fits your product positioning. For details, please see the TXBeautyManager.h file.

    Beauty Filter Style Description
    TXBeautyStyleSmooth The smooth style, which features more obvious skin smoothing effects and is suitable for live showrooms
    TXBeautyStyleNature The natural style, which retains more facial details and is more natural
    TXBeautyStylePitu The Pitu style, which uses the beauty filter algorithm developed by YouTu Lab. Its effect is between the smooth style and the natural style, that is, it retains more skin details than the smooth style and delivers more obvious skin smoothing effects than the natural style.

    You can call the setBeautyStyle API in TXBeautyManager to set the beauty filter style.

    ItemConfigurationDescription
    Beauty filter strength Via the setBeautyLevel API in `TXBeautyManager` Value range: 0-9. `0` means the filter is disabled. The greater the value, the more obvious the effect.
    Skin brightening filter strength Via the setWhitenessLevel API in `TXBeautyManager` Value range: 0-9. `0` means the filter is disabled. The greater the value, the more obvious the effect.
    Rosy skin filter strength Via the setRuddyLevel API in `TXBeautyManager` Value range: 0-9. `0` means the filter is disabled. The greater the value, the more obvious the effect.

    9. Set color filters

    • Call getBeautyManager in V2TXLivePusher to get a TXBeautyManager instance to set color filters.
    • Call the setFilter API in TXBeautyManager to set color filters. Color filters are a technology that adjusts the color tone of sections of an image. For example, it may lighten the yellow sections of an image to achieve the effect of skin brightening, or add warm tones to a video to give it a refreshing and soft boost.
    • Call the setFilterStrength API in TXBeautyManager to set the strength of a color filter. The higher the strength, the more obvious the effect.

    Based on our experience of operating Now Live, it’s not enough to use the setBeautyStyle API in TXBeautyManager to set the beauty filter style. The setBeautyStyle API must be used together with setFilter in order to produce richer effects. Given this, our designers have developed 17 color filters and integrated them into the demo.

    NSString * path = [[NSBundle mainBundle] pathForResource:@"FilterResource" ofType:@"bundle"];
    path = [path stringByAppendingPathComponent:lookupFileName];
    UIImage *image = [UIImage imageWithContentsOfFile:path];
    [[_pusher getBeautyManager] setFilter:image];
    [[_pusher getBeautyManager] setFilterStrength:0.5f];
    

    10. Manage devices

    V2TXLivePusher provides a series of APIs for the control of devices. You can call getDeviceManager to get a TXDeviceManager instance for device management. For detailed instructions, please see TXDeviceManager API.

    11. Set the mirror effect for audience

    Call setRenderMirror in V2TXLivePusher to set the camera mirror mode, which affects the way video images are presented to audience. By default, the local image seen by the host is flipped when the front camera is used.

    12. Publish streams in landscape mode

    In most cases, hosts stream while holding their phones vertically, and audience watch videos in portrait resolutions (e.g., 540 × 960). However, there are also cases where hosts hold phones horizontally, and ideally, audience should watch videos in landscape resolutions (960 × 540).

    By default, V2TXLivePusher outputs videos in portrait resolutions. You can publish landscape-mode videos to audience by modifying a parameter of the setVideoQuality API.

    [_pusher setVideoQuality:videoQuality
                resolutionMode:isLandscape ? V2TXLiveVideoResolutionModeLandscape : V2TXLiveVideoResolutionModePortrait];
    

    13. Set audio effects

    Call getAudioEffectManager in V2TXLivePusher to get a TXAudioEffectManager instance, which can be used to mix background music and set in-ear monitoring, reverb, and other audio effects. Background music mixing means mixing into the published stream the music played by the host’s phone so that audience can also hear the music.

    • Call the enableVoiceEarMonitor API in TXAudioEffectManager to enable in-ear monitoring, which allows hosts to hear their vocals in earphones when they sing.
    • Call the setVoiceReverbType API in TXAudioEffectManager to add reverb effects such as karaoke, hall, husky, and metal. The effects are applied to the videos watched by audience.
    • Call the setVoiceChangerType API in TXAudioEffectManager to add voice changing effects such as little girl and middle-aged man to enrich host-audience interaction. The effects are applied to the videos watched by audience.
    Note:

    For detailed instructions, please see TXAudioEffectManager API.

    14. Set watermarks

    Call setWatermark in V2TXLivePusher to add a watermark to videos output by the SDK. The position of the watermark is determined by the (x, y, scale) parameter passed in.

    • The watermark image must be in PNG rather than JPG format. The former carries opacity information, which allows the SDK to better address the image aliasing issue (changing the extension of a JPG image to PNG won’t work).
    • The (x, y, scale) parameter specifies the normalized coordinates of the watermark relative to the resolution of the published video. For example, if the resolution of the published video is 540 x 960, and (x, y, scale) is set to (0.1, 0.1, 0.1), the actual pixel coordinates of the watermark will be (540 x 0.1, 960 x 0.1). The width of the watermark will be the video width x 0.1, and the height will be scaled automatically.
    // Set a video watermark
    [_pusher setWatermark:[UIImage imageNamed:@"watermark"] x:0.03 y:0.015 scale:1];
    

    15. Inform hosts of poor network conditions

    Connecting phones to Wi-Fi does not necessarily guarantee network conditions. In case of poor Wi-Fi signal or limited bandwidth, the network speed of a Wi-Fi connected phone may be slower than that of a phone using 4G. Hosts should be informed when their network conditions are bad and be prompted to switch to a different network.

    You can capture the V2TXLIVE_WARNING_NETWORK_BUSY event using onWarning in V2TXLivePusherObserver. The event indicates poor network conditions for hosts, which result in stuttering for audience. When this event occurs, you can send a UI message about poor network conditions to hosts, as shown above.

    - (void)onWarning:(V2TXLiveCode)code
    message:(NSString *)msg
    extraInfo:(NSDictionary *)extraInfo {
    dispatch_async(dispatch_get_main_queue(), ^{
    if (code == V2TXLIVE_WARNING_NETWORK_BUSY) {
    [_notification displayNotificationWithMessage:
    @"Your network conditions are poor. Please switch to a different network." forDuration:5];
    }
    });
    }

    Event Handling

    Listening for events

    The SDK listens for publishing events and errors via the V2TXLivePusherObserver delegate. See V2TXLiveCode for a detailed list of events and error codes.

    Errors

    An error indicates that the SDK encountered a serious problem that made it impossible for stream publishing to continue.

    Event ID Code Description
    V2TXLIVE_ERROR_FAILED -1 A common error not yet classified
    V2TXLIVE_ERROR_INVALID_PARAMETER -2 An invalid parameter was passed in during API calling.
    V2TXLIVE_ERROR_REFUSED -3 The API call was rejected.
    V2TXLIVE_ERROR_NOT_SUPPORTED -4 The API cannot be called.
    V2TXLIVE_ERROR_INVALID_LICENSE -5 Failed to call the API due to invalid license.
    V2TXLIVE_ERROR_REQUEST_TIMEOUT -6 The server request timed out.
    V2TXLIVE_ERROR_SERVER_PROCESS_FAILED -7 The server could not handle your request.

    Warnings

    A warning indicates that the SDK encountered a problem whose severity level is warning. Warning events trigger tentative protection or recovery logic and can often be resolved.

    Event ID Code Description
    V2TXLIVE_WARNING_NETWORK_BUSY 1101 Bad network connection: data upload blocked due to limited upstream bandwidth.
    V2TXLIVE_WARNING_VIDEO_BLOCK 2105 Latency during video playback.
    V2TXLIVE_WARNING_CAMERA_START_FAILED -1301 Failed to turn the camera on.
    V2TXLIVE_WARNING_CAMERA_OCCUPIED -1316 The camera is occupied. Try a different camera.
    V2TXLIVE_WARNING_CAMERA_NO_PERMISSION -1314 No access to the camera. This usually occurs on mobile devices and may be because the user denied the access.
    V2TXLIVE_WARNING_MICROPHONE_START_FAILED -1302 Failed to turn the mic on.
    V2TXLIVE_WARNING_MICROPHONE_OCCUPIED -1319 The mic is occupied. This occurs when, for example, the user is having a call on the mobile device.
    V2TXLIVE_WARNING_MICROPHONE_NO_PERMISSION -1317 No access to the mic. This usually occurs on mobile devices and may be because the user denied the access.
    V2TXLIVE_WARNING_SCREEN_CAPTURE_NOT_SUPPORTED -1309 The system does not support screen sharing.
    V2TXLIVE_WARNING_SCREEN_CAPTURE_START_FAILED -1308 Failed to start screen recording. If this occurs on a mobile device, it may be because the user denied the access.
    V2TXLIVE_WARNING_SCREEN_CAPTURE_INTERRUPTED -7001 Screen recording was stopped by the system.