宜飞思工业设计网站,科技微网站,微信微信,网站建设做什么科目WebRTC 系列#xff08;二、本地 demo#xff0c;H5、Android、iOS#xff09;
上一篇博客中#xff0c;我已经展示了各端的本地 demo#xff0c;大家应该知道 WebRTC 怎么用了。在本地 demo 中是用了一个 RemotePeerConnection 来模拟远端#xff0c;可能理解起来还有点…WebRTC 系列二、本地 demoH5、Android、iOS
上一篇博客中我已经展示了各端的本地 demo大家应该知道 WebRTC 怎么用了。在本地 demo 中是用了一个 RemotePeerConnection 来模拟远端可能理解起来还有点麻烦下面就来实现点对点通话这个 demo 完成后流程会更加清晰。
一、信令服务器
既然不同端之间要通信那就需要一个中间人来做桥梁传递通信链路建立之前的信息也就是 offer、answer、iceCandidate 这些信息。信令服务器的实现手段也有很多可以通过 SocketIO、WebSocket、Netty 等。
这里我就选择用 Java 通过 WebSocket 搭建一个信令服务器了后续可能还会写个 nodejs 版的。
在 Android Studio 中新建一个项目然后在项目中创建一个 Java Module到时候就可以在 Java Module 中运行 main 方法了这样就不用再下载一个 IDEA 了。
Java Module 的 build 中添加 WebSocket 依赖
plugins {id java-library
}java {sourceCompatibility JavaVersion.VERSION_1_7targetCompatibility JavaVersion.VERSION_1_7
}dependencies {// WebSocketimplementation org.java-websocket:Java-WebSocket:1.5.3
}
然后编写 WebSocket 服务端代码
package com.qinshou.webrtcdemo_server;import org.java_websocket.WebSocket;
import org.java_websocket.handshake.ClientHandshake;
import org.java_websocket.server.WebSocketServer;import java.net.InetSocketAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.List;public class WebSocketServerHelper {private WebSocketServer mWebSocketServer;private final ListWebSocket mWebSockets new ArrayList();private static final String HOST_NAME 192.168.1.105;private static final int PORT 8888;public void start() {InetSocketAddress inetSocketAddress new InetSocketAddress(HOST_NAME, PORT);mWebSocketServer new WebSocketServer(inetSocketAddress) {Overridepublic void onOpen(WebSocket conn, ClientHandshake handshake) {System.out.println(onOpen--- conn);// 客户端连接时保存到集合中mWebSockets.add(conn);}Overridepublic void onClose(WebSocket conn, int code, String reason, boolean remote) {System.out.println(onClose--- conn);// 客户端断开时从集合中移除mWebSockets.remove(conn);}Overridepublic void onMessage(WebSocket conn, String message) {
// System.out.println(onMessage--- message);// 消息直接透传给除发送方以外的连接for (WebSocket webSocket : mWebSockets) {if (webSocket ! conn) {webSocket.send(message);}}}Overridepublic void onError(WebSocket conn, Exception ex) {System.out.println(onError--- conn , ex--- ex);// 客户端连接异常时从集合中移除mWebSockets.remove(conn);}Overridepublic void onStart() {System.out.println(onStart);}};mWebSocketServer.start();}public void stop() {if (mWebSocketServer null) {return;}for (WebSocket webSocket : mWebSocketServer.getConnections()) {webSocket.close();}try {mWebSocketServer.stop();} catch (InterruptedException e) {throw new RuntimeException(e);}mWebSocketServer null;}public static void main(String[] args) {new WebSocketServerHelper().start();}
}
p2p 通信场景下信令服务器不需要做太多只需要分发消息即可为了简单我也没有引入用户和房间等概念所以在测试的时候只能连接两个客户端。
二、消息格式
既然我们需要将 sdp 和 iceCandidate 传递给别人那双方就得约定一个格式这样传递给对方后对方才能解析p2p 阶段我们只需要定义 sdp 和 iceCandidate 消息即可其中 sdp
// sdp
{msgType: sdp,type: sessionDescription.type,sdp: sessionDescription.sdp
}// iceCandidate
{msgType: iceCandidate,id: iceCandidate.sdpMid,label: iceCandidate.sdpMLineIndex,candidate: iceCandidate.candidate
}
三、H5
代码与 local_demo 其实差不了太多只是要将模拟远端的 RemotePeerConnection 去掉在主动呼叫或收到 offer 时创建一个 PeerConnection 就可以。然后把发送 sdp、iceCandidate 的地方改成通过 WebSocket 发送即可所以我们还需要创建一个 WebSocket 客户端。
1.添加依赖
WebSocket 也是 H5 的标准之一所以不需要我们额外引入。
2.p2p_demo.html
htmlheadtitleP2P Demo/titlestylebody {overflow: hidden;margin: 0px;padding: 0px;}#local_view {width: 100%;height: 100%;}#remote_view {width: 9%;height: 16%;position: absolute;top: 10%;right: 10%;}#left {width: 10%;height: 5%;position: absolute;left: 10%;top: 10%;}#p_websocket_state,#input_server_url,.my_button {width: 100%;height: 100%;display: block;margin-bottom: 10%;}/style
/headbodyvideo idlocal_view autoplay controls muted/videovideo idremote_view autoplay controls muted/videodiv idleftp idp_websocket_stateWebSocket 已断开/pinput idinput_server_url typetext placeholder请输入服务器地址 valuews://192.168.1.105:8888/inputbutton idbtn_connect classmy_button onclickconnect()连接 WebSocket/buttonbutton idbtn_disconnect classmy_button onclickdisconnect()断开 WebSocket/buttonbutton idbtn_call classmy_button onclickcall()呼叫/buttonbutton idbtn_hang_up classmy_button onclickhangUp()挂断/button/div
/bodyscript typetext/javascriptlet localView document.getElementById(local_view);let remoteView document.getElementById(remote_view);var localStream;var peerConnection;function createPeerConnection() {let rtcPeerConnection new RTCPeerConnection();rtcPeerConnection.oniceconnectionstatechange function (event) {if (disconnected event.target.iceConnectionState) {hangUp();}}rtcPeerConnection.onicecandidate function (event) {console.log(onicecandidate--- event.candidate);let iceCandidate event.candidate;if (iceCandidate null) {return;}sendIceCandidate(iceCandidate);}rtcPeerConnection.ontrack function (event) {console.log(remote ontrack--- event.streams);let streams event.streams;if (streams streams.length 0) {remoteView.srcObject streams[0];}}return rtcPeerConnection}function call() {// 创建 PeerConnectionpeerConnection createPeerConnection();// 为 PeerConnection 添加音轨、视轨for (let i 0; localStream ! null i localStream.getTracks().length; i) {const track localStream.getTracks()[i];peerConnection.addTrack(track, localStream);}// 通过 PeerConnection 创建 offer获取 sdppeerConnection.createOffer().then(function (sessionDescription) {console.log(create offer success.);// 将 offer sdp 作为参数 setLocalDescriptionpeerConnection.setLocalDescription(sessionDescription).then(function () {console.log(set local sdp success.);// 发送 offer sdpsendOffer(sessionDescription)})})}function sendOffer(offer) {var jsonObject {msgType: sdp,type: offer.type,sdp: offer.sdp};send(JSON.stringify(jsonObject));}function receivedOffer(offer) {// 创建 PeerConnectionpeerConnection createPeerConnection();// 为 PeerConnection 添加音轨、视轨for (let i 0; localStream ! null i localStream.getTracks().length; i) {const track localStream.getTracks()[i];peerConnection.addTrack(track, localStream);}// 将 offer sdp 作为参数 setRemoteDescriptionpeerConnection.setRemoteDescription(offer).then(function () {console.log(set remote sdp success.);// 通过 PeerConnection 创建 answer获取 sdppeerConnection.createAnswer().then(function (sessionDescription) {console.log(create answer success.);// 将 answer sdp 作为参数 setLocalDescriptionpeerConnection.setLocalDescription(sessionDescription).then(function () {console.log(set local sdp success.);// 发送 answer sdpsendAnswer(sessionDescription);})})})}function sendAnswer(answer) {var jsonObject {msgType: sdp,type: answer.type,sdp: answer.sdp};send(JSON.stringify(jsonObject));}function receivedAnswer(answer) {// 收到 answer sdp将 answer sdp 作为参数 setRemoteDescriptionpeerConnection.setRemoteDescription(answer).then(function () {console.log(set remote sdp success.);})}function sendIceCandidate(iceCandidate) {var jsonObject {msgType: iceCandidate,id: iceCandidate.sdpMid,label: iceCandidate.sdpMLineIndex,candidate: iceCandidate.candidate};send(JSON.stringify(jsonObject));}function receivedCandidate(iceCandidate) {peerConnection.addIceCandidate(iceCandidate);}function hangUp() {if (peerConnection ! null) {peerConnection.close();peerConnection null;}remoteView.removeAttribute(src);remoteView.load();}navigator.mediaDevices.getUserMedia({ audio: true, video: true }).then(function (mediaStream) {// 初始化 PeerConnectionFactory// 创建 EglBase// 创建 PeerConnectionFactory// 创建音轨// 创建视轨localStream mediaStream;// 初始化本地视频渲染控件// 初始化远端视频渲染控件// 开始本地渲染localView.srcObject mediaStream;}).catch(function (error) {console.log(error--- error);})/scriptscript typetext/javascriptvar websocket;function connect() {let inputServerUrl document.getElementById(input_server_url);let pWebsocketState document.getElementById(p_websocket_state);let url inputServerUrl.value;websocket new WebSocket(url);websocket.onopen function () {console.log(onOpen);pWebsocketState.innerText WebSocket 已连接;}websocket.onmessage function (message) {console.log(onmessage--- message.data);let jsonObject JSON.parse(message.data);let msgType jsonObject[msgType];if (sdp msgType) {let type jsonObject[type];if (offer type) {let options {type: jsonObject[type],sdp: jsonObject[sdp]}let offer new RTCSessionDescription(options);receivedOffer(offer);} else if (answer type) {let options {type: jsonObject[type],sdp: jsonObject[sdp]}let answer new RTCSessionDescription(options);receivedAnswer(answer);}} else if (iceCandidate msgType) {let options {sdpMLineIndex: jsonObject[label],sdpMid: jsonObject[id],candidate: jsonObject[candidate]}let iceCandidate new RTCIceCandidate(options);receivedCandidate(iceCandidate);}}websocket.onclose function (error) {console.log(onclose--- error);pWebsocketState.innerText WebSocket 已断开;}websocket.onerror function (error) {console.log(onerror--- error);}}function disconnect() {websocket.close();}function send(message) {if (!websocket) {return;}websocket.send(message);}/script/html
主要流程都是一样的有什么不懂的地方可以留言由于需要 p2p 通话至少需要两个端我们就等所有端都实现了再最后任选两个端来看效果。
四、Android
1.添加依赖
Android 则需要在 app 的 build.gradle 中引入 WebSocket 依赖
// WebSocket
implementation org.java-websocket:Java-WebSocket:1.5.3
权限申请跟之前的一样就不重复了。
2.布局
?xml version1.0 encodingutf-8?
androidx.constraintlayout.widget.ConstraintLayout xmlns:androidhttp://schemas.android.com/apk/res/androidxmlns:apphttp://schemas.android.com/apk/res-autoxmlns:toolshttp://schemas.android.com/toolsandroid:layout_widthmatch_parentandroid:layout_heightmatch_parentandroid:background#FF000000android:keepScreenOntruetools:context.P2PDemoActivityorg.webrtc.SurfaceViewRendererandroid:idid/svr_localandroid:layout_widthmatch_parentandroid:layout_height0dpapp:layout_constraintBottom_toBottomOfparentapp:layout_constraintDimensionRatio9:16app:layout_constraintEnd_toEndOfparentapp:layout_constraintStart_toStartOfparentapp:layout_constraintTop_toTopOfparent /org.webrtc.SurfaceViewRendererandroid:idid/svr_remoteandroid:layout_width90dpandroid:layout_height0dpandroid:layout_marginTop30dpandroid:layout_marginEnd30dpapp:layout_constraintDimensionRatio9:16app:layout_constraintEnd_toEndOfparentapp:layout_constraintTop_toTopOfparent /androidx.appcompat.widget.LinearLayoutCompatandroid:layout_widthmatch_parentandroid:layout_heightwrap_contentandroid:layout_marginStart30dpandroid:layout_marginTop30dpandroid:layout_marginEnd30dpandroid:orientationverticalapp:layout_constraintStart_toStartOfparentapp:layout_constraintTop_toTopOfparentandroidx.appcompat.widget.AppCompatTextViewandroid:idid/tv_websocket_stateandroid:layout_widthmatch_parentandroid:layout_heightwrap_contentandroid:textWebSocketServer 已断开android:textColor#FFFFFFFF /androidx.appcompat.widget.AppCompatEditTextandroid:idid/et_server_urlandroid:layout_widthmatch_parentandroid:layout_heightwrap_contentandroid:hint请输入服务器地址android:textColor#FFFFFFFFandroid:textColorHint#FFFFFFFF /androidx.appcompat.widget.AppCompatButtonandroid:idid/btn_connectandroid:layout_widthwrap_contentandroid:layout_heightwrap_contentandroid:text连接 WebSocketandroid:textAllCapsfalse /androidx.appcompat.widget.AppCompatButtonandroid:idid/btn_disconnectandroid:layout_widthwrap_contentandroid:layout_heightwrap_contentandroid:text断开 WebSocketandroid:textAllCapsfalse /androidx.appcompat.widget.AppCompatButtonandroid:idid/btn_callandroid:layout_widthwrap_contentandroid:layout_heightwrap_contentandroid:text呼叫 /androidx.appcompat.widget.AppCompatButtonandroid:idid/btn_hang_upandroid:layout_widthwrap_contentandroid:layout_heightwrap_contentandroid:text挂断 //androidx.appcompat.widget.LinearLayoutCompat
/androidx.constraintlayout.widget.ConstraintLayout
3.P2PDemoActivity
package com.qinshou.webrtcdemo_android;import android.content.Context;
import android.os.Bundle;
import android.text.TextUtils;
import android.view.View;
import android.widget.EditText;
import android.widget.TextView;import androidx.appcompat.app.AppCompatActivity;import org.json.JSONException;
import org.json.JSONObject;
import org.webrtc.AudioSource;
import org.webrtc.AudioTrack;
import org.webrtc.Camera2Capturer;
import org.webrtc.Camera2Enumerator;
import org.webrtc.CameraEnumerator;
import org.webrtc.DataChannel;
import org.webrtc.DefaultVideoDecoderFactory;
import org.webrtc.DefaultVideoEncoderFactory;
import org.webrtc.EglBase;
import org.webrtc.IceCandidate;
import org.webrtc.MediaConstraints;
import org.webrtc.MediaStream;
import org.webrtc.PeerConnection;
import org.webrtc.PeerConnectionFactory;
import org.webrtc.RtpReceiver;
import org.webrtc.SessionDescription;
import org.webrtc.SurfaceTextureHelper;
import org.webrtc.SurfaceViewRenderer;
import org.webrtc.VideoCapturer;
import org.webrtc.VideoDecoderFactory;
import org.webrtc.VideoEncoderFactory;
import org.webrtc.VideoSource;
import org.webrtc.VideoTrack;import java.util.ArrayList;
import java.util.List;/*** Author: MrQinshou* Email: cqflqinhao126.com* Date: 2023/3/21 17:22* Description: P2P demo*/
public class P2PDemoActivity extends AppCompatActivity {private static final String TAG P2PDemoActivity.class.getSimpleName();private static final String AUDIO_TRACK_ID ARDAMSa0;private static final String VIDEO_TRACK_ID ARDAMSv0;private static final ListString STREAM_IDS new ArrayListString() {{add(ARDAMS);}};private static final String SURFACE_TEXTURE_HELPER_THREAD_NAME SurfaceTextureHelperThread;private static final int WIDTH 1280;private static final int HEIGHT 720;private static final int FPS 30;private EglBase mEglBase;private PeerConnectionFactory mPeerConnectionFactory;private VideoCapturer mVideoCapturer;private AudioTrack mAudioTrack;private VideoTrack mVideoTrack;private PeerConnection mPeerConnection;private WebSocketClientHelper mWebSocketClientHelper new WebSocketClientHelper();Overrideprotected void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.activity_p2p_demo);((EditText) findViewById(R.id.et_server_url)).setText(ws://192.168.1.105:8888);findViewById(R.id.btn_connect).setOnClickListener(new View.OnClickListener() {Overridepublic void onClick(View view) {String url ((EditText) findViewById(R.id.et_server_url)).getText().toString().trim();mWebSocketClientHelper.connect(url);}});findViewById(R.id.btn_disconnect).setOnClickListener(new View.OnClickListener() {Overridepublic void onClick(View view) {mWebSocketClientHelper.disconnect();}});findViewById(R.id.btn_call).setOnClickListener(new View.OnClickListener() {Overridepublic void onClick(View view) {call();}});findViewById(R.id.btn_hang_up).setOnClickListener(new View.OnClickListener() {Overridepublic void onClick(View view) {hangUp();}});mWebSocketClientHelper.setOnWebSocketListener(new WebSocketClientHelper.OnWebSocketClientListener() {Overridepublic void onOpen() {runOnUiThread(new Runnable() {Overridepublic void run() {((TextView) findViewById(R.id.tv_websocket_state)).setText(WebSocket 已连接);}});}Overridepublic void onClose() {runOnUiThread(new Runnable() {Overridepublic void run() {((TextView) findViewById(R.id.tv_websocket_state)).setText(WebSocket 已断开);}});}Overridepublic void onMessage(String message) {try {JSONObject jsonObject new JSONObject(message);String msgType jsonObject.optString(msgType);if (TextUtils.equals(sdp, msgType)) {String type jsonObject.optString(type);if (TextUtils.equals(offer, type)) {String sdp jsonObject.optString(sdp);SessionDescription offer new SessionDescription(SessionDescription.Type.OFFER, sdp);receivedOffer(offer);} else if (TextUtils.equals(answer, type)) {String sdp jsonObject.optString(sdp);SessionDescription answer new SessionDescription(SessionDescription.Type.ANSWER, sdp);receivedAnswer(answer);}} else if (TextUtils.equals(iceCandidate, msgType)) {String id jsonObject.optString(id);int label jsonObject.optInt(label);String candidate jsonObject.optString(candidate);IceCandidate iceCandidate new IceCandidate(id, label, candidate);receivedCandidate(iceCandidate);}} catch (JSONException e) {e.printStackTrace();}}});// 初始化 PeerConnectionFactoryinitPeerConnectionFactory(P2PDemoActivity.this);// 创建 EglBasemEglBase EglBase.create();// 创建 PeerConnectionFactorymPeerConnectionFactory createPeerConnectionFactory(mEglBase);// 创建音轨mAudioTrack createAudioTrack(mPeerConnectionFactory);// 创建视轨mVideoCapturer createVideoCapturer();VideoSource videoSource createVideoSource(mPeerConnectionFactory, mVideoCapturer);mVideoTrack createVideoTrack(mPeerConnectionFactory, videoSource);// 初始化本地视频渲染控件这个方法非常重要不初始化会黑屏SurfaceViewRenderer svrLocal findViewById(R.id.svr_local);svrLocal.init(mEglBase.getEglBaseContext(), null);mVideoTrack.addSink(svrLocal);// 初始化远端视频渲染控件这个方法非常重要不初始化会黑屏SurfaceViewRenderer svrRemote findViewById(R.id.svr_remote);svrRemote.init(mEglBase.getEglBaseContext(), null);// 开始本地渲染// 创建 SurfaceTextureHelper用来表示 camera 初始化的线程SurfaceTextureHelper surfaceTextureHelper SurfaceTextureHelper.create(SURFACE_TEXTURE_HELPER_THREAD_NAME, mEglBase.getEglBaseContext());// 初始化视频采集器mVideoCapturer.initialize(surfaceTextureHelper, P2PDemoActivity.this, videoSource.getCapturerObserver());mVideoCapturer.startCapture(WIDTH, HEIGHT, FPS);}Overrideprotected void onDestroy() {super.onDestroy();if (mEglBase ! null) {mEglBase.release();mEglBase null;}if (mVideoCapturer ! null) {try {mVideoCapturer.stopCapture();} catch (InterruptedException e) {e.printStackTrace();}mVideoCapturer.dispose();mVideoCapturer null;}if (mAudioTrack ! null) {mAudioTrack.dispose();mAudioTrack null;}if (mVideoTrack ! null) {mVideoTrack.dispose();mVideoTrack null;}if (mPeerConnection ! null) {mPeerConnection.close();mPeerConnection null;}SurfaceViewRenderer svrLocal findViewById(R.id.svr_local);svrLocal.release();SurfaceViewRenderer svrRemote findViewById(R.id.svr_remote);svrRemote.release();mWebSocketClientHelper.disconnect();}private void initPeerConnectionFactory(Context context) {PeerConnectionFactory.initialize(PeerConnectionFactory.InitializationOptions.builder(context).createInitializationOptions());}private PeerConnectionFactory createPeerConnectionFactory(EglBase eglBase) {VideoEncoderFactory videoEncoderFactory new DefaultVideoEncoderFactory(eglBase.getEglBaseContext(), true, true);VideoDecoderFactory videoDecoderFactory new DefaultVideoDecoderFactory(eglBase.getEglBaseContext());return PeerConnectionFactory.builder().setVideoEncoderFactory(videoEncoderFactory).setVideoDecoderFactory(videoDecoderFactory).createPeerConnectionFactory();}private AudioTrack createAudioTrack(PeerConnectionFactory peerConnectionFactory) {AudioSource audioSource peerConnectionFactory.createAudioSource(new MediaConstraints());AudioTrack audioTrack peerConnectionFactory.createAudioTrack(AUDIO_TRACK_ID, audioSource);audioTrack.setEnabled(true);return audioTrack;}private VideoCapturer createVideoCapturer() {VideoCapturer videoCapturer null;CameraEnumerator cameraEnumerator new Camera2Enumerator(P2PDemoActivity.this);for (String deviceName : cameraEnumerator.getDeviceNames()) {// 前摄像头if (cameraEnumerator.isFrontFacing(deviceName)) {videoCapturer new Camera2Capturer(P2PDemoActivity.this, deviceName, null);}}return videoCapturer;}private VideoSource createVideoSource(PeerConnectionFactory peerConnectionFactory, VideoCapturer videoCapturer) {// 创建视频源VideoSource videoSource peerConnectionFactory.createVideoSource(videoCapturer.isScreencast());return videoSource;}private VideoTrack createVideoTrack(PeerConnectionFactory peerConnectionFactory, VideoSource videoSource) {// 创建视轨VideoTrack videoTrack peerConnectionFactory.createVideoTrack(VIDEO_TRACK_ID, videoSource);videoTrack.setEnabled(true);return videoTrack;}private PeerConnection createPeerConnection() {PeerConnection.RTCConfiguration rtcConfiguration new PeerConnection.RTCConfiguration(new ArrayList());PeerConnection peerConnection mPeerConnectionFactory.createPeerConnection(rtcConfiguration, new PeerConnection.Observer() {Overridepublic void onSignalingChange(PeerConnection.SignalingState signalingState) {}Overridepublic void onIceConnectionChange(PeerConnection.IceConnectionState iceConnectionState) {if (iceConnectionState PeerConnection.IceConnectionState.DISCONNECTED) {runOnUiThread(new Runnable() {Overridepublic void run() {hangUp();}});}}Overridepublic void onIceConnectionReceivingChange(boolean b) {}Overridepublic void onIceGatheringChange(PeerConnection.IceGatheringState iceGatheringState) {}Overridepublic void onIceCandidate(IceCandidate iceCandidate) {ShowLogUtil.verbose(onIceCandidate--- iceCandidate);sendIceCandidate(iceCandidate);}Overridepublic void onIceCandidatesRemoved(IceCandidate[] iceCandidates) {}Overridepublic void onAddStream(MediaStream mediaStream) {ShowLogUtil.verbose(onAddStream--- mediaStream);if (mediaStream null || mediaStream.videoTracks null || mediaStream.videoTracks.isEmpty()) {return;}runOnUiThread(new Runnable() {Overridepublic void run() {SurfaceViewRenderer svrRemote findViewById(R.id.svr_remote);mediaStream.videoTracks.get(0).addSink(svrRemote);}});}Overridepublic void onRemoveStream(MediaStream mediaStream) {}Overridepublic void onDataChannel(DataChannel dataChannel) {}Overridepublic void onRenegotiationNeeded() {}Overridepublic void onAddTrack(RtpReceiver rtpReceiver, MediaStream[] mediaStreams) {}});return peerConnection;}private void call() {// 创建 PeerConnectionmPeerConnection createPeerConnection();// 为 PeerConnection 添加音轨、视轨mPeerConnection.addTrack(mAudioTrack, STREAM_IDS);mPeerConnection.addTrack(mVideoTrack, STREAM_IDS);// 通过 PeerConnection 创建 offer获取 sdpMediaConstraints mediaConstraints new MediaConstraints();mPeerConnection.createOffer(new MySdpObserver() {Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose(create offer success.);// 将 offer sdp 作为参数 setLocalDescriptionmPeerConnection.setLocalDescription(new MySdpObserver() {Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}Overridepublic void onSetSuccess() {ShowLogUtil.verbose(set local sdp success.);// 发送 offer sdpsendOffer(sessionDescription);}}, sessionDescription);}Overridepublic void onSetSuccess() {}}, mediaConstraints);}private void sendOffer(SessionDescription offer) {try {JSONObject jsonObject new JSONObject();jsonObject.put(msgType, sdp);jsonObject.put(type, offer);jsonObject.put(sdp, offer.description);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void receivedOffer(SessionDescription offer) {// 创建 PeerConnectionmPeerConnection createPeerConnection();// 为 PeerConnection 添加音轨、视轨mPeerConnection.addTrack(mAudioTrack, STREAM_IDS);mPeerConnection.addTrack(mVideoTrack, STREAM_IDS);// 将 offer sdp 作为参数 setRemoteDescriptionmPeerConnection.setRemoteDescription(new MySdpObserver() {Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}Overridepublic void onSetSuccess() {ShowLogUtil.verbose(set remote sdp success.);// 通过 PeerConnection 创建 answer获取 sdpMediaConstraints mediaConstraints new MediaConstraints();mPeerConnection.createAnswer(new MySdpObserver() {Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {ShowLogUtil.verbose(create answer success.);// 将 answer sdp 作为参数 setLocalDescriptionmPeerConnection.setLocalDescription(new MySdpObserver() {Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}Overridepublic void onSetSuccess() {ShowLogUtil.verbose(set local sdp success.);// 发送 answer sdpsendAnswer(sessionDescription);}}, sessionDescription);}Overridepublic void onSetSuccess() {}}, mediaConstraints);}}, offer);}private void sendAnswer(SessionDescription answer) {try {JSONObject jsonObject new JSONObject();jsonObject.put(msgType, sdp);jsonObject.put(type, answer);jsonObject.put(sdp, answer.description);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void receivedAnswer(SessionDescription answer) {// 收到 answer sdp将 answer sdp 作为参数 setRemoteDescriptionmPeerConnection.setRemoteDescription(new MySdpObserver() {Overridepublic void onCreateSuccess(SessionDescription sessionDescription) {}Overridepublic void onSetSuccess() {ShowLogUtil.verbose(set remote sdp success.);}}, answer);}private void sendIceCandidate(IceCandidate iceCandidate) {try {JSONObject jsonObject new JSONObject();jsonObject.put(msgType, iceCandidate);jsonObject.put(id, iceCandidate.sdpMid);jsonObject.put(label, iceCandidate.sdpMLineIndex);jsonObject.put(candidate, iceCandidate.sdp);mWebSocketClientHelper.send(jsonObject.toString());} catch (JSONException e) {e.printStackTrace();}}private void receivedCandidate(IceCandidate iceCandidate) {mPeerConnection.addIceCandidate(iceCandidate);}private void hangUp() {// 关闭 PeerConnectionif (mPeerConnection ! null) {mPeerConnection.close();mPeerConnection.dispose();mPeerConnection null;}// 释放远端视频渲染控件SurfaceViewRenderer svrRemote findViewById(R.id.svr_remote);svrRemote.clearImage();}
}
其中 WebSocketClientHelper 也只是对 WebSocket 的一个简单封装
package com.qinshou.webrtcdemo_android;import org.java_websocket.client.WebSocketClient;
import org.java_websocket.handshake.ServerHandshake;import java.net.URI;/*** Author: MrQinshou* Email: cqflqinhao126.com* Date: 2023/2/8 9:33* Description: 类描述*/
public class WebSocketClientHelper {public interface OnWebSocketClientListener {void onOpen();void onClose();void onMessage(String message);}private WebSocketClient mWebSocketClient;private OnWebSocketClientListener mOnWebSocketClientListener new OnWebSocketClientListener() {Overridepublic void onOpen() {}Overridepublic void onClose() {}Overridepublic void onMessage(String message) {}};public void setOnWebSocketListener(OnWebSocketClientListener onWebSocketClientListener) {if (onWebSocketClientListener null) {return;}mOnWebSocketClientListener onWebSocketClientListener;}public void connect(String url) {mWebSocketClient new WebSocketClient(URI.create(url)) {Overridepublic void onOpen(ServerHandshake handshakedata) {ShowLogUtil.debug(onOpen);mOnWebSocketClientListener.onOpen();}Overridepublic void onMessage(String message) {
// ShowLogUtil.debug(onMessage--- message);mOnWebSocketClientListener.onMessage(message);}Overridepublic void onClose(int code, String reason, boolean remote) {ShowLogUtil.debug(onClose--- code);mOnWebSocketClientListener.onClose();}Overridepublic void onError(Exception ex) {ShowLogUtil.debug(onError);}};mWebSocketClient.connect();}public void disconnect() {if (mWebSocketClient null) {return;}mWebSocketClient.close();}public void send(String message) {if (mWebSocketClient null) {return;}mWebSocketClient.send(message);}
}跟 H5 是一样的有什么不懂的地方可以留言由于需要 p2p 通话至少需要两个端我们就等所有端都实现了再最后任选两个端来看效果。
五、iOS
1.添加依赖
iOS 也需要在 app 的 build.gradle 中引入 WebSocket 依赖
...
target WebRTCDemo-iOS do...pod Starscream, ~ 4.0.0
end
...
权限申请跟之前的一样就不重复了。
2.P2PViewController
//
// LocalDemoViewController.swift
// WebRTCDemo-iOS
//
// Created by 覃浩 on 2023/3/21.
//import UIKit
import WebRTC
import SnapKitclass P2PDemoViewController: UIViewController {private static let AUDIO_TRACK_ID ARDAMSa0private static let VIDEO_TRACK_ID ARDAMSv0private static let STREAM_IDS [ARDAMS]private static let WIDTH 1280private static let HEIGHT 720private static let FPS 30private var localView: RTCEAGLVideoView!private var remoteView: RTCEAGLVideoView!private var peerConnectionFactory: RTCPeerConnectionFactory!private var audioTrack: RTCAudioTrack?private var videoTrack: RTCVideoTrack?/**iOS 需要将 Capturer 保存为全局变量否则无法渲染本地画面*/private var videoCapturer: RTCVideoCapturer?/**iOS 需要将远端流保存为全局变量否则无法渲染远端画面*/private var remoteStream: RTCMediaStream?private var peerConnection: RTCPeerConnection?private var lbWebSocketState: UILabel? nilprivate var tfServerUrl: UITextField? nilprivate let webSocketHelper WebSocketClientHelper()override func viewDidLoad() {super.viewDidLoad()// 表明 View 不要扩展到整个屏幕而是在 NavigationBar 下的区域edgesForExtendedLayout UIRectEdge()self.view.backgroundColor UIColor.black// WebSocket 状态文本框lbWebSocketState UILabel()lbWebSocketState!.textColor UIColor.whitelbWebSocketState!.text WebSocket 已断开self.view.addSubview(lbWebSocketState!)lbWebSocketState!.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.right.equalToSuperview().offset(-30)make.height.equalTo(40)})// 服务器地址输入框tfServerUrl UITextField()tfServerUrl!.textColor UIColor.whitetfServerUrl!.text ws://192.168.1.105:8888tfServerUrl!.placeholder 请输入服务器地址tfServerUrl!.delegate selfself.view.addSubview(tfServerUrl!)tfServerUrl!.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.right.equalToSuperview().offset(-30)make.height.equalTo(20)make.top.equalTo(lbWebSocketState!.snp.bottom).offset(10)})// 连接 WebSocket 按钮let btnConnect UIButton()btnConnect.backgroundColor UIColor.lightGraybtnConnect.setTitle(连接 WebSocket, for: .normal)btnConnect.setTitleColor(UIColor.black, for: .normal)btnConnect.addTarget(self, action: #selector(connect), for: .touchUpInside)self.view.addSubview(btnConnect)btnConnect.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(140)make.height.equalTo(40)make.top.equalTo(tfServerUrl!.snp.bottom).offset(10)})// 断开 WebSocket 按钮let btnDisconnect UIButton()btnDisconnect.backgroundColor UIColor.lightGraybtnDisconnect.setTitle(断开 WebSocket, for: .normal)btnDisconnect.setTitleColor(UIColor.black, for: .normal)btnDisconnect.addTarget(self, action: #selector(disconnect), for: .touchUpInside)self.view.addSubview(btnDisconnect)btnDisconnect.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(140)make.height.equalTo(40)make.top.equalTo(btnConnect.snp.bottom).offset(10)})// 呼叫按钮let btnCall UIButton()btnCall.backgroundColor UIColor.lightGraybtnCall.setTitle(呼叫, for: .normal)btnCall.setTitleColor(UIColor.black, for: .normal)btnCall.addTarget(self, action: #selector(call), for: .touchUpInside)self.view.addSubview(btnCall)btnCall.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(80)make.height.equalTo(40)make.top.equalTo(btnDisconnect.snp.bottom).offset(10)})// 挂断按钮let btnHangUp UIButton()btnHangUp.backgroundColor UIColor.lightGraybtnHangUp.setTitle(挂断, for: .normal)btnHangUp.setTitleColor(UIColor.black, for: .normal)btnHangUp.addTarget(self, action: #selector(hangUp), for: .touchUpInside)self.view.addSubview(btnHangUp)btnHangUp.snp.makeConstraints({ make inmake.left.equalToSuperview().offset(30)make.width.equalTo(80)make.height.equalTo(40)make.top.equalTo(btnCall.snp.bottom).offset(10)})webSocketHelper.setDelegate(delegate: self)// 初始化 PeerConnectionFactoryinitPeerConnectionFactory()// 创建 EglBase// 创建 PeerConnectionFactorypeerConnectionFactory createPeerConnectionFactory()// 创建音轨audioTrack createAudioTrack(peerConnectionFactory: peerConnectionFactory)// 创建视轨videoTrack createVideoTrack(peerConnectionFactory: peerConnectionFactory)let tuple createVideoCapturer(videoSource: videoTrack!.source)let captureDevice tuple.captureDevicevideoCapturer tuple.videoCapture// 初始化本地视频渲染控件localView RTCEAGLVideoView()localView.delegate selfself.view.insertSubview(localView,at: 0)localView.snp.makeConstraints({ make inmake.width.equalToSuperview()make.height.equalTo(localView.snp.width).multipliedBy(16.0/9.0)make.centerY.equalToSuperview()})videoTrack?.add(localView!)// 初始化远端视频渲染控件remoteView RTCEAGLVideoView()remoteView.delegate selfself.view.insertSubview(remoteView, aboveSubview: localView)remoteView.snp.makeConstraints({ make inmake.width.equalTo(90)make.height.equalTo(160)make.top.equalToSuperview().offset(30)make.right.equalToSuperview().offset(-30)})// 开始本地渲染(videoCapturer as? RTCCameraVideoCapturer)?.startCapture(with: captureDevice!, format: captureDevice!.activeFormat, fps: P2PDemoViewController.FPS)}override func viewDidDisappear(_ animated: Bool) {(videoCapturer as? RTCCameraVideoCapturer)?.stopCapture()videoCapturer nilpeerConnection?.close()peerConnection nilwebSocketHelper.disconnect()}private func initPeerConnectionFactory() {RTCPeerConnectionFactory.initialize()}private func createPeerConnectionFactory() - RTCPeerConnectionFactory {var videoEncoderFactory RTCDefaultVideoEncoderFactory()var videoDecoderFactory RTCDefaultVideoDecoderFactory()if TARGET_OS_SIMULATOR ! 0 {videoEncoderFactory RTCSimluatorVideoEncoderFactory()videoDecoderFactory RTCSimulatorVideoDecoderFactory()}return RTCPeerConnectionFactory(encoderFactory: videoEncoderFactory, decoderFactory: videoDecoderFactory)}private func createAudioTrack(peerConnectionFactory: RTCPeerConnectionFactory) - RTCAudioTrack {let mandatoryConstraints : [String : String] [:]let optionalConstraints : [String : String] [:]let audioSource peerConnectionFactory.audioSource(with: RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints))let audioTrack peerConnectionFactory.audioTrack(with: audioSource, trackId: P2PDemoViewController.AUDIO_TRACK_ID)audioTrack.isEnabled truereturn audioTrack}private func createVideoTrack(peerConnectionFactory: RTCPeerConnectionFactory) - RTCVideoTrack? {let videoSource peerConnectionFactory.videoSource()let videoTrack peerConnectionFactory.videoTrack(with: videoSource, trackId: P2PDemoViewController.VIDEO_TRACK_ID)videoTrack.isEnabled truereturn videoTrack}private func createVideoCapturer(videoSource: RTCVideoSource) - (captureDevice: AVCaptureDevice?, videoCapture: RTCVideoCapturer?) {let videoCapturer RTCCameraVideoCapturer(delegate: videoSource)let captureDevices RTCCameraVideoCapturer.captureDevices()if (captureDevices.count 0) {return (nil, nil)}var captureDevice: AVCaptureDevice?for c in captureDevices {// 前摄像头if (c.position .front) {captureDevice cbreak}}if (captureDevice nil) {return (nil, nil)}return (captureDevice, videoCapturer)}private func createPeerConnection() - RTCPeerConnection {let rtcConfiguration RTCConfiguration()let mandatoryConstraints : [String : String] [:]let optionalConstraints : [String : String] [:]let mediaConstraints RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)let peerConnection peerConnectionFactory.peerConnection(with: rtcConfiguration, constraints: mediaConstraints, delegate: self)return peerConnection}objc private func call() {// 创建 PeerConnectionpeerConnection createPeerConnection()// 为 PeerConnection 添加音轨、视轨peerConnection?.add(audioTrack!, streamIds: P2PDemoViewController.STREAM_IDS)peerConnection?.add(videoTrack!, streamIds: P2PDemoViewController.STREAM_IDS)// 通过 PeerConnection 创建 offer获取 sdplet mandatoryConstraints: [String : String] [:]let optionalConstraints: [String : String] [:]let mediaConstraints RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)peerConnection?.offer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose(create offer success.)// 将 offer sdp 作为参数 setLocalDescriptionself.peerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose(set local sdp success.)// 发送 offer sdpself.sendOffer(offer: sessionDescription!)})})}private func sendOffer(offer: RTCSessionDescription) {var jsonObject [String : String]()jsonObject[msgType] sdpjsonObject[type] offerjsonObject[sdp] offer.sdpdo {let data try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose(error---\(error))}}private func receivedOffer(offer: RTCSessionDescription) {// 创建 PeerConnectionpeerConnection createPeerConnection()// 为 PeerConnection 添加音轨、视轨peerConnection?.add(audioTrack!, streamIds: P2PDemoViewController.STREAM_IDS)peerConnection?.add(videoTrack!, streamIds: P2PDemoViewController.STREAM_IDS)// 将 offer sdp 作为参数 setRemoteDescriptionpeerConnection?.setRemoteDescription(offer, completionHandler: { _ inShowLogUtil.verbose(set remote sdp success.)// 通过 PeerConnection 创建 answer获取 sdplet mandatoryConstraints : [String : String] [:]let optionalConstraints : [String : String] [:]let mediaConstraints RTCMediaConstraints(mandatoryConstraints: mandatoryConstraints, optionalConstraints: optionalConstraints)self.peerConnection?.answer(for: mediaConstraints, completionHandler: { sessionDescription, error inShowLogUtil.verbose(create answer success.)// 将 answer sdp 作为参数 setLocalDescriptionself.peerConnection?.setLocalDescription(sessionDescription!, completionHandler: { _ inShowLogUtil.verbose(set local sdp success.)// 发送 answer sdpself.sendAnswer(answer: sessionDescription!)})})})}private func sendAnswer(answer: RTCSessionDescription) {var jsonObject [String : String]()jsonObject[msgType] sdpjsonObject[type] answerjsonObject[sdp] answer.sdpdo {let data try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose(error---\(error))}}private func receivedAnswer(answer: RTCSessionDescription) {// 收到 answer sdp将 answer sdp 作为参数 setRemoteDescriptionpeerConnection?.setRemoteDescription(answer, completionHandler: { _ in ShowLogUtil.verbose(set remote sdp success.)})}private func sendIceCandidate(iceCandidate: RTCIceCandidate) {var jsonObject [String : Any]()jsonObject[msgType] iceCandidatejsonObject[id] iceCandidate.sdpMidjsonObject[label] iceCandidate.sdpMLineIndexjsonObject[candidate] iceCandidate.sdpdo {let data try JSONSerialization.data(withJSONObject: jsonObject)webSocketHelper.send(message: String(data: data, encoding: .utf8)!)} catch {ShowLogUtil.verbose(error---\(error))}}private func receivedCandidate(iceCandidate: RTCIceCandidate) {peerConnection?.add(iceCandidate)}objc private func hangUp() {// 关闭 PeerConnectionpeerConnection?.close()peerConnection nil// 释放远端视频渲染控件if let track remoteStream?.videoTracks.first {track.remove(remoteView!)}}objc private func connect() {webSocketHelper.connect(url: tfServerUrl!.text!.trimmingCharacters(in: .whitespacesAndNewlines))}objc private func disconnect() {webSocketHelper.disconnect()}
}// MARK: - RTCVideoViewDelegate
extension P2PDemoViewController: RTCVideoViewDelegate {func videoView(_ videoView: RTCVideoRenderer, didChangeVideoSize size: CGSize) {}
}// MARK: - RTCPeerConnectionDelegate
extension P2PDemoViewController: RTCPeerConnectionDelegate {func peerConnection(_ peerConnection: RTCPeerConnection, didChange stateChanged: RTCSignalingState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didAdd stream: RTCMediaStream) {ShowLogUtil.verbose(peerConnection didAdd stream---\(stream))DispatchQueue.main.async {self.remoteStream streamif let track stream.videoTracks.first {track.add(self.remoteView!)}if let audioTrack stream.audioTracks.first{audioTrack.source.volume 8}}}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove stream: RTCMediaStream) {}func peerConnectionShouldNegotiate(_ peerConnection: RTCPeerConnection) {}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceConnectionState) {if (newState .disconnected) {DispatchQueue.main.async {self.hangUp()}}}func peerConnection(_ peerConnection: RTCPeerConnection, didChange newState: RTCIceGatheringState) {}func peerConnection(_ peerConnection: RTCPeerConnection, didGenerate candidate: RTCIceCandidate) {ShowLogUtil.verbose(didGenerate candidate---\(candidate))self.sendIceCandidate(iceCandidate: candidate)}func peerConnection(_ peerConnection: RTCPeerConnection, didRemove candidates: [RTCIceCandidate]) {}func peerConnection(_ peerConnection: RTCPeerConnection, didOpen dataChannel: RTCDataChannel) {}
}// MARK: - UITextFieldDelegate
extension P2PDemoViewController: UITextFieldDelegate {func textFieldShouldReturn(_ textField: UITextField) - Bool {textField.resignFirstResponder()return true}
}// MARK: - WebSocketDelegate
extension P2PDemoViewController: WebSocketDelegate {func onOpen() {lbWebSocketState?.text WebSocket 已连接}func onClose() {lbWebSocketState?.text WebSocket 已断开}func onMessage(message: String) {do {let data message.data(using: .utf8)let jsonObject: [String : Any] try JSONSerialization.jsonObject(with: data!) as! [String : Any]let msgType jsonObject[msgType] as? Stringif (sdp msgType) {let type jsonObject[type] as? Stringif (offer type) {let sdp jsonObject[sdp] as! Stringlet offer RTCSessionDescription(type: .offer, sdp: sdp)receivedOffer(offer: offer)} else if (answer type) {let sdp jsonObject[sdp] as! Stringlet answer RTCSessionDescription(type: .answer, sdp: sdp)receivedAnswer(answer: answer)}} else if (iceCandidate msgType) {let id jsonObject[id] as? Stringlet label jsonObject[label] as? Int32let candidate jsonObject[candidate] as? Stringlet iceCandidate RTCIceCandidate(sdp: candidate!, sdpMLineIndex: label!, sdpMid: id)receivedCandidate(iceCandidate: iceCandidate)}} catch {}}
}其中 WebSocketClientHelper 也只是对 WebSocket 的一个简单封装
//
// WebClientHelper.swift
// WebRTCDemo-iOS
//
// Created by 覃浩 on 2023/3/1.
//import Starscreampublic protocol WebSocketDelegate {func onOpen()func onClose()func onMessage(message: String)
}class WebSocketClientHelper {private var webSocket: WebSocket?private var delegate: WebSocketDelegate?func setDelegate(delegate: WebSocketDelegate) {self.delegate delegate}func connect(url: String) {let request URLRequest(url: URL(string: url)!)webSocket WebSocket(request: request)webSocket?.onEvent { event inswitch event {case .connected(let headers):self.delegate?.onOpen()breakcase .disconnected(let reason, let code):self.delegate?.onClose()breakcase .text(let string):self.delegate?.onMessage(message: string)breakcase .binary(let data):breakcase .ping(_):breakcase .pong(_):breakcase .viabilityChanged(_):breakcase .reconnectSuggested(_):breakcase .cancelled:self.delegate?.onClose()breakcase .error(let error):self.delegate?.onClose()break}}webSocket?.connect()}func disconnect() {webSocket?.disconnect()}func send(message: String) {webSocket?.write(string: message)}
}好了现在三端都实现了我们可以来看看效果了。
六、效果展示
运行 WebSocketServerHelper 的 main() 方法我们可以看到服务端已经开启 运行 html、Android、iOS 三端任选其中两端连接 WebSocket这两端任选一端点击呼叫 需要注意的是我在 WebSocket 中没有引入用户和房间的概念呼叫都是透传给除自己外的所有连接所以在测试的时候只能连接两个客户端不用的时候就要断开 WebSocket。
七、总结
实现完成后可以感觉到点对点呼叫其实也没有多难跟本地 Demo 的流程大致一样只是我们需要将音视频通话的协商信息通过网络传输而已所以我之前才说明白 WebRTC 的流程比较重要信令服务器反而在其次毕竟真实场景中信令服务器还会加入很多业务逻辑。
下一次我们在信令服务器中加入一些逻辑来实现多人通话。
八、Demo
Demo 传送门
