This document provides a detailed description that makes it easy for Native project developers to debug and integrate the APIs for Game Multimedia Engine (GME).
This document only provides the major APIs to help you get started with GME. You can refer to the Demo to debug and integrate the APIs.
GME provides two services: voice chat service and voice message and speech-to-text service, both of which rely on key APIs such as Init and Poll.
Note on Init APIIf you need to use voice chat and voice message services at the same time, you only need to call
Init
API once.
Initializing GMEAPI: Init
Calling Poll periodically to trigger event callbacksAPI: Poll
In the SDK download guide page, download the client’s Demo project codesDownLoad.
import com.tencent.TMG.ITMGContext;
import com.tencent.av.sig.AuthBuffer;
import com.tencent.bugly.crashreport.CrashReport;
To use the voice feature, get the ITMGContext
object first.
public static ITMGContext GetInstance(Context context)
//MainActivity.java
import com.tencent.TMG.ITMGContext;
ITMGContext tmgContext = ITMGContext.GetInstance(this);
The API class uses the Delegate
method to send callback notifications to the application. Register the callback function to the SDK for receiving callback messages before room entry.
Register the callback function to the SDK for receiving callback messages before room entry.
//ITMGContext
public abstract int SetTMGDelegate(ITMGDelegate delegate);
//MainActivity.java
tmgContext.SetTMGDelegate(TMGCallbackDispatcher.getInstance());
Override this callback function in the constructor to process the parameters of the callback.
//MainActivity.java
tmgContext.SetTMGDelegate(TMGCallbackDispatcher.getInstance());
//RealTimeVoiceActivity.java
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Processing callbacks
}
}
// Refer to TMGCallbackDispatcher.java, TMGCallbackHelper.java, and TMGDispatcherBase.java
Parameter | Type | Description |
---|---|---|
type | ITMGContext.ITMG_MAIN_EVENT_TYPE | Event type in the callback response |
data | Intent message type | Callback message, i.e., event data |
sdkAppId
parameter, please see Voice Service Activation Guide.public abstract int Init(String sdkAppId, String openId);
Parameter | Type | Description |
---|---|---|
sdkAppId | String | AppId provided by the GME service from the Tencent Cloud Console |
OpenId | String | OpenId can only be in Int64 type, which is passed after being converted to a string. |
//MainActivity.java
int nRet = tmgContext.Init(appId, openId);
if (nRet == AV_OK )
{
GMEAuthBufferHelper.getInstance().setGEMParams(appId, key, openId);
// Step 4/11: Poll to trigger callback
//https://intl.cloud.tencent.com/document/product/607/15210?from_cn_redirect=1#.E8.A7.A6.E5.8F.91.E4.BA.8B.E4.BB.B6.E5.9B.9E.E8.B0.83
EnginePollHelper.createEnginePollHelper();
showToast("Init success");
}else if (nRet == AV_ERR_HAS_IN_THE_STATE) // SDK has been initialized. This operation is successful.
{
showToast("Init success");
}else
{
showToast("Init error errorCode:" + nRet);
}
Event callbacks can be triggered by periodically calling the Poll
API in update
. The Poll
API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.
You can refer to the EnginePollHelper.java
file in the demo.
//MainActivity.java
[EnginePollHelper createEnginePollHelper];
//EnginePollHelper.java
private Handler mhandler = new Handler();
private Runnable mRunnable = new Runnable() {
@Override
public void run() {
if (s_pollEnabled) {
if (ITMGContext.GetInstance(null) != null)
ITMGContext.GetInstance(null).Poll();
}
mhandler.postDelayed(mRunnable, 33);
}
};
// For the code of calling Poll periodically, please see EnginePollHelper.java.
Generate AuthBuffer
for encryption and authentication of relevant features.
To get authentication for voice message and speech-to-text, the room ID parameter must be set to null
.
AuthBuffer public native byte[] genAuthBuffer(int sdkAppId, String roomId, String openId, String key)
Parameter | Type | Description |
---|---|---|
appId | int | AppId from the Tencent Cloud console. |
roomId | string | Room ID, which can contain up to 127 characters (For voice message, enter "null".) |
openId | string | User ID, which is the same as openId during initialization. |
key | string | Permission key from the Tencent Cloud console. |
//GMEAuthBufferHelper.java
import com.tencent.av.sig.AuthBuffer;// Header file
public byte[] createAuthBuffer(String roomId)
{
byte[] authBuffer;
// Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment,
// please use the backend deployment key as detailed in https://intl.cloud.tencent.com/document/product/607/12218
if (TextUtils.isEmpty(roomId))
{
authBuffer = AuthBuffer.getInstance().genAuthBuffer(Integer.parseInt(mAppId), "0", mOpenId, mKey);
}else
{
authBuffer = AuthBuffer.getInstance().genAuthBuffer(Integer.parseInt(mAppId), roomId, mOpenId, mKey);
}
return authBuffer;
}
When a user enters a room with the generated authentication information, the ITMG_MAIN_EVENT_TYPE_ENTER_ROOM
message will be received as a callback. Mic and speaker are not enabled by default after room entry. The returned value of AV_OK
indicates a success.
For more information on how to choose a room audio type, please see Sound Quality Selection.
public abstract int EnterRoom(String roomID, int roomType, byte[] authBuffer);
Parameter | Type | Description |
---|---|---|
roomId | String | Room ID, which can contain up to 127 characters |
roomType | int | Room audio type |
authBuffer | byte[] | Authentication code |
//RealTimeVoiceActivity.java
byte[] authBuffer = GMEAuthBufferHelper.getInstance().createAuthBuffer(roomId);
ITMGContext.GetInstance(this).EnterRoom(roomId, roomType, authBuffer);
After the user enters the room, the message ITMG_MAIN_EVENT_TYPE_ENTER_ROOM
will be sent and identified in the OnEvent
function for callback and processing. A successful callback means that the room entry is successful, and the billing starts.
//RealTimeVoiceActivity.java
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Step 6/11 : Perform the enter room event
int nErrCode = TMGCallbackHelper.ParseIntentParams2(data).nErrCode;
String strMsg = TMGCallbackHelper.ParseIntentParams2(data).strErrMsg;
if (nErrCode == AV_OK)
{
appendLog2MonitorView("EnterRomm success");
}else
{
appendLog2MonitorView(String.format(Locale.getDefault(), "EnterRomm errCode:%d errMsg:%s", nErrCode, strMsg));
}
}
}
Error Code Value | Cause and Suggested Solution |
---|---|
7006 | Authentication failed. Possible causes:
|
7007 | Already in another room. |
1001 | The user was already in the process of entering a room but repeated this operation. It is recommended not to call the room entering API until the room entry callback is returned. |
1003 | The user was already in the room and called the room entering API again. |
1101 | Make sure that the SDK is initialized, `openId` complies with the rules, the APIs are called in the same thread, and the `Poll` API is called normally. |
This API is used to enable/disable the mic. Mic and speaker are not enabled by default after room entry.
//RealTimeVoiceActivity.java
ITMGContext.GetInstance(this).GetAudioCtrl().EnableMic(true);
This API is used to enable/disable the speaker.
//RealTimeVoiceActivity.java
ITMGContext.GetInstance(this).GetAudioCtrl().EnableSpeaker(true);
This API is called to exit the current room. It is an async API. The returned value of AV_OK
indicates a successful async delivery.
//RealTimeVoiceActivity.java
ITMGContext.GetInstance(this).ExitRoom();
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM
. The sample code is shown below:
//RealTimeVoiceActivity.java
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_EXIT_ROOM == type)
{
// Receive the event of successful room exit
}
}
Call authentication initialization after initializing the SDK. For more information on how to get the authBuffer
, please see genAuthBuffer
(the voice chat authentication information API).
public abstract int ApplyPTTAuthbuffer(byte[] authBuffer);
Parameter | Type | Description |
---|---|---|
authBuffer | String | Authentication |
//VoiceMessageRecognitionActivity.java
byte[] authBuffer = GMEAuthBufferHelper.getInstance().createAuthBuffer("");
ITMGContext.GetInstance(this).GetPTT().ApplyPTTAuthbuffer(authBuffer);
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation. To stop recording, call StopRecording
. The callback will be returned after the recording is stopped.
public abstract int StartRecordingWithStreamingRecognition (String filePath);
public abstract int StartRecordingWithStreamingRecognition (String filePath,String language,String translatelanguage);
public abstract int StopRecording();
Parameter | Type | Description |
---|---|---|
filePath | String | Path of stored audio file |
speechLanguage | String | The language in which the audio file is to be converted to text. For parameters, please see Language Parameter Reference List |
translateLanguage | String | The language into which the audio file will be translated. For parameters, please see Language Parameter Reference List (This parameter is currently unavailable. Enter the same value as that of speechLanguage ) |
//VoiceMessageRecognitionActivity.java
ITMGContext.GetInstance(this).GetPTT().StartRecordingWithStreamingRecognition(recordfilePath,"cmn-Hans-CN");
After streaming speech recognition is started, you need to listen for callback messages in the callback function onEvent
. The event message is ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE
, namely returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.
The event message will be identified in the OnEvent function
based on the actual needs. The passed parameters include the following four messages.
Message Name | Description |
---|---|
result | A return code for judging whether the streaming speech recognition is successful. |
text | Text converted from speech |
file_path | Local path of stored recording file |
file_id | Backend URL address of recording file, which will be retained for 90 days |
//VoiceMessageRecognitionActivity.java
import static com.tencent.TMG.ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE;
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (type == ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE)
{
// Step 1.3/3 handle ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE event
mIsRecording = false;
if (nErrCode ==0)
{
String recordfilePath = data.getStringExtra("file_path");
mRecFilePathView.setText(recordfilePath);
String recordFileUrl = data.getStringExtra("file_id");
mRecFileUrlView.setText(recordFileUrl);
}
else
{
appendLog2MonitorView("Record and recognition fail errCode:" + nErrCode);
}
}
}
Error Code | Description | Suggested Solution |
---|---|---|
32775 | Streaming speech-to-text conversion failed, but recording succeeded. | Call the `UploadRecordedFile` API to upload the recording file and then call the `SpeechToText` API to perform speech-to-text conversion. |
32777 | Streaming speech-to-text converting failed, but recording and upload succeeded | The message returned contains a backend URL after successful upload. Call the `SpeechToText` API to perform speech-to-text conversion. |
32786 | Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
public abstract int StopRecording();
//VoiceMessageRecognitionActivity.java
ITMGContext.GetInstance(this).GetPTT().StopRecording();
Was this page helpful?