Native SDK Quick Access

Last updated: 2021-06-22 15:05:07

    This document provides a detailed description that makes it easy for Native project developers to debug and integrate the APIs for Game Multimedia Engine (GME).

    This document only provides the major APIs to help you get started with GME. You can refer to the Demo to debug and integrate the APIs.

    Key Considerations for Using GME

    GME provides two services: voice chat service and voice message and speech-to-text service, both of which rely on key APIs such as Init and Poll.

    Note on Init API

    If you need to use voice chat and voice message services at the same time, you only need to call Init API once.

    API call flowchart

    image

    Directions

    Key APIs

    Initializing GMEAPI: Init
    Calling Poll periodically to trigger event callbacksAPI: Poll

    Voice Chat

    Voice Message

    Uninitializing GMEAPI: UnInit

    Key API Access

    Downloading demo

    In the SDK download guide page, download the client’s Demo project codesDownLoad.

    2. Importing the header file

    import com.tencent.TMG.ITMGContext;
    import com.tencent.av.sig.AuthBuffer;
    import com.tencent.bugly.crashreport.CrashReport;

    3. Getting singleton

    To use the voice feature, get the ITMGContext object first.

    Function prototype

    public static ITMGContext GetInstance(Context context)

    Sample code

    //MainActivity.java
    import com.tencent.TMG.ITMGContext;
    ITMGContext tmgContext = ITMGContext.GetInstance(this);

    4. Setting callback

    The API class uses the Delegate method to send callback notifications to the application. Register the callback function to the SDK for receiving callback messages before room entry.

    Function prototype and sample code

    Register the callback function to the SDK for receiving callback messages before room entry.

    //ITMGContext
    public abstract int SetTMGDelegate(ITMGDelegate delegate);

    //MainActivity.java
    tmgContext.SetTMGDelegate(TMGCallbackDispatcher.getInstance());

    Callback examples

    Override this callback function in the constructor to process the parameters of the callback.

    //MainActivity.java
    tmgContext.SetTMGDelegate(TMGCallbackDispatcher.getInstance());

    //RealTimeVoiceActivity.java
    public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
    if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
    {
    // Processing callbacks
    }
    }

    // Refer to TMGCallbackDispatcher.java, TMGCallbackHelper.java, and TMGDispatcherBase.java
    Parameter Type Description
    type ITMGContext.ITMG_MAIN_EVENT_TYPE Event type in the callback response
    data Intent message type Callback message, i.e., event data

    5. Initializing SDK

    • This API is used to initialize the GME service. It is recommended to call it when initializing the application.
    • For more information on how to get the sdkAppId parameter, please see Voice Service Activation Guide.
    • The openID uniquely identifies a user with the rules stipulated by the application developer and unique in the application (currently, only INT64 is supported).

    Function prototype

    public abstract int Init(String sdkAppId, String openId);
    Parameter Type Description
    sdkAppId String AppId provided by the GME service from the Tencent Cloud Console
    OpenId String OpenId can only be in Int64 type, which is passed after being converted to a string.

    Sample code

    //MainActivity.java
    int nRet = tmgContext.Init(appId, openId);
    if (nRet == AV_OK )
    {
    GMEAuthBufferHelper.getInstance().setGEMParams(appId, key, openId);
    // Step 4/11: Poll to trigger callback
    //https://intl.cloud.tencent.com/document/product/607/15210?from_cn_redirect=1#.E8.A7.A6.E5.8F.91.E4.BA.8B.E4.BB.B6.E5.9B.9E.E8.B0.83
    EnginePollHelper.createEnginePollHelper();
    showToast("Init success")
    ;
    }else if (nRet == AV_ERR_HAS_IN_THE_STATE) // SDK has been initialized. This operation is successful.
    {
    showToast("Init success")
    ;
    }else
    {
    showToast("Init error errorCode:" + nRet)
    ;
    }

    6. Triggering event callback

    Event callbacks can be triggered by periodically calling the Poll API in update. The Poll API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.

    You can refer to the EnginePollHelper.java file in the demo.

    Sample code

    //MainActivity.java
    [EnginePollHelper createEnginePollHelper];

    //EnginePollHelper.java
    private Handler mhandler = new Handler();
    private Runnable mRunnable = new Runnable() {
    @Override
    public void run() {
    if (s_pollEnabled)
    {
    if (ITMGContext.GetInstance(null) !
    = null)
    ITMGContext.GetInstance(null).Poll();
    }
    mhandler.postDelayed(mRunnable, 33);
    }
    };
    // For the code of calling Poll periodically, please see EnginePollHelper.java.

    7. Authentication

    Generate AuthBuffer for encryption and authentication of relevant features.
    To get authentication for voice message and speech-to-text, the room ID parameter must be set to null.

    Function prototype

    AuthBuffer public native byte[] genAuthBuffer(int sdkAppId, String roomId, String openId, String key)
    Parameter Type Description
    appId int AppId from the Tencent Cloud console.
    roomId string Room ID, which can contain up to 127 characters (For voice message, enter "null".)
    openId string User ID, which is the same as openId during initialization.
    key string Permission key from the Tencent Cloud console.

    Sample code

    //GMEAuthBufferHelper.java
    import com.tencent.av.sig.AuthBuffer;// Header file
    public byte[] createAuthBuffer(String roomId)
    {
    byte[] authBuffer;
    // Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment,
    // please use the backend deployment key as detailed in https://intl.cloud.tencent.com/document/product/607/12218
    if (TextUtils.isEmpty(roomId))
    {
    authBuffer = AuthBuffer.getInstance().genAuthBuffer(Integer.parseInt(mAppId), "0", mOpenId, mKey);
    }else
    {
    authBuffer = AuthBuffer.getInstance().genAuthBuffer(Integer.parseInt(mAppId), roomId, mOpenId, mKey);
    }
    return authBuffer;
    }

    Voice Chat Access

    1. Entering a room

    When a user enters a room with the generated authentication information, the ITMG_MAIN_EVENT_TYPE_ENTER_ROOM message will be received as a callback. Mic and speaker are not enabled by default after room entry. The returned value of AV_OK indicates a success.

    For more information on how to choose a room audio type, please see Sound Quality Selection.

    Function prototype

    public abstract int EnterRoom(String roomID, int roomType, byte[] authBuffer);
    Parameter Type Description
    roomId String Room ID, which can contain up to 127 characters
    roomType int Room audio type
    authBuffer byte[] Authentication code

    Sample code

    //RealTimeVoiceActivity.java
    byte[] authBuffer = GMEAuthBufferHelper.getInstance().createAuthBuffer(roomId);
    ITMGContext.GetInstance(this).EnterRoom(roomId, roomType, authBuffer);

    Callback for room entry

    After the user enters the room, the message ITMG_MAIN_EVENT_TYPE_ENTER_ROOM will be sent and identified in the OnEvent function for callback and processing. A successful callback means that the room entry is successful, and the billing starts. It will be free of charge if the total call duration of the day is less than 700 minutes.

    • Sample code
      Sample code for processing the callback, including room entry and network disconnection events.
      //RealTimeVoiceActivity.java
      public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
      if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
      {
      // Step 6/11 : Perform the enter room event
      int nErrCode = TMGCallbackHelper.ParseIntentParams2(data).nErrCode;
      String strMsg = TMGCallbackHelper.ParseIntentParams2(data).strErrMsg;
      if (nErrCode == AV_OK)
      {
      appendLog2MonitorView("EnterRomm success")
      ;
      }else
      {
      appendLog2MonitorView(String.format(Locale.getDefault(), "EnterRomm errCode:%d errMsg:%s", nErrCode, strMsg))
      ;
      }
      }
      }
    • Error code
      Error Code Value Cause and Suggested Solution
      7006 Authentication failed. Possible causes:
      • The `AppID` does not exist or is incorrect.
      • An error occurred while authenticating the `authbuff`.
      • Authentication expired.
      • The `openId` does not meet the specification.
      7007 Already in another room.
      1001 The user was already in the process of entering a room but repeated this operation. It is recommended not to call the room entering API until the room entry callback is returned.
      1003 The user was already in the room and called the room entering API again.
      1101 Make sure that the SDK is initialized, `openId` complies with the rules, the APIs are called in the same thread, and the `Poll` API is called normally.

    2. Enabling or disabling the microphone

    This API is used to enable/disable the mic. Mic and speaker are not enabled by default after room entry.

    Sample code

    //RealTimeVoiceActivity.java
    ITMGContext.GetInstance(this).GetAudioCtrl().EnableMic(true);

    3. Enabling or disabling the speaker

    This API is used to enable/disable the speaker.

    Sample code

    //RealTimeVoiceActivity.java
    ITMGContext.GetInstance(this).GetAudioCtrl().EnableSpeaker(true);

    4. Exiting the room

    This API is called to exit the current room. It is an async API. The returned value of AV_OK indicates a successful async delivery.

    Sample code

    //RealTimeVoiceActivity.java
    ITMGContext.GetInstance(this).ExitRoom();

    Callback for room exit

    After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM. The sample code is shown below:

    //RealTimeVoiceActivity.java
    public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
    if (ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_EXIT_ROOM == type)
    {
    // Receive the event of successful room exit
    }
    }

    Voice Message Access

    1. Initializing authentication

    Call authentication initialization after initializing the SDK. For more information on how to get the authBuffer, please see genAuthBuffer (the voice chat authentication information API).

    Function prototype

    public abstract int ApplyPTTAuthbuffer(byte[] authBuffer);
    Parameter Type Description
    authBuffer String Authentication

    Sample code

    //VoiceMessageRecognitionActivity.java
    byte[] authBuffer = GMEAuthBufferHelper.getInstance().createAuthBuffer("");
    ITMGContext.GetInstance(this).GetPTT().ApplyPTTAuthbuffer(authBuffer);

    2. Starting streaming speech recognition

    This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation. To stop recording, call StopRecording. The callback will be returned after the recording is stopped.

    Function prototype

    public abstract int StartRecordingWithStreamingRecognition (String filePath);
    public abstract int StartRecordingWithStreamingRecognition (String filePath,String language,String translatelanguage)
    ;
    public abstract int StopRecording()
    ;
    Parameter Type Description
    filePath String Path of stored audio file
    speechLanguage String The language in which the audio file is to be converted to text. For parameters, please see Language Parameter Reference List
    translateLanguage String The language into which the audio file will be translated. For parameters, please see Language Parameter Reference List (This parameter is currently unavailable. Enter the same value as that of speechLanguage)

    Sample code

    //VoiceMessageRecognitionActivity.java
    ITMGContext.GetInstance(this).GetPTT().StartRecordingWithStreamingRecognition(recordfilePath,"cmn-Hans-CN");

    Callback for streaming speech recognition

    After streaming speech recognition is started, you need to listen for callback messages in the callback function onEvent. The event message is ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE, namely returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.

    The event message will be identified in the OnEvent function based on the actual needs. The passed parameters include the following four messages.

    Message Name Description
    result A return code for judging whether the streaming speech recognition is successful.
    text Text converted from speech
    file_path Local path of stored recording file
    file_id Backend URL address of recording file, which will be retained for 90 days
    • Sample code
      //VoiceMessageRecognitionActivity.java
      import static com.tencent.TMG.ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE;
      public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data)
      {
      if (type == ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE)
      {
      // Step 1.3/3 handle ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE event
      mIsRecording = false;
      if (nErrCode ==0)
      {
      String recordfilePath = data.getStringExtra("file_path");
      mRecFilePathView.setText(recordfilePath);

      String recordFileUrl = data.getStringExtra("file_id");
      mRecFileUrlView.setText(recordFileUrl);
      }
      else
      {
      appendLog2MonitorView("Record and recognition fail errCode:" + nErrCode)
      ;
      }
      }

      }
    • Error code
      Error Code Description Suggested Solution
      32775 Streaming speech-to-text conversion failed, but recording succeeded. Call the `UploadRecordedFile` API to upload the recording file and then call the `SpeechToText` API to perform speech-to-text conversion.
      32777 Streaming speech-to-text converting failed, but recording and upload succeeded The message returned contains a backend URL after successful upload. Call the `SpeechToText` API to perform speech-to-text conversion.
      32786 Streaming speech-to-text conversion failed. During streaming recording, wait for the execution result of the streaming recording API to return.

    3. Stopping recording

    This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.

    Function prototype

    public abstract int StopRecording();

    Sample code

    //VoiceMessageRecognitionActivity.java
    ITMGContext.GetInstance(this).GetPTT().StopRecording();