Today we are going to work not on one app, but on two apps. The first one will run on the mobile phone and will send data to the second one and the second app will run on our computers and will display the received information on screen. To be more specific, we will be able to be on screen how the phone is moving in reality. All these using ARCore.

Just as you have done yesterday, you will need to download the source code, which you will need to complete in order to make the two apps work. So download the two archives, extract them and open both of them in Unity. Just as you did yesterday.

This time, however, we are going to modify two scripts: one on the mobile phone app (Phone Movement) and the other on the PC app (Track Phone). So the PC app displays the position and orientation of the mobile phone in real life. So, the PC app will receive data and the phone app will send the necessary data to be displayed. As a result, the script we are going to update for the mobile phone app is called UDPSend.cs and thescript we are going to update for the computer app is called UDPReceive.cs. UDP is a communication protocol used in computer programming in order to make possible the communication of two computers. UDP is not the only communication protocol used nowadays, but it is one of the easiest protocols to implement. However, even though it is easy to implement it, UDP has a great setback: we need to have both our devices in the same network. As a result MAKE SURE THAT BOTH YOUR COMPUTER AND YOUR PHONE ARE CONNECTED TO THE SAME WI-FI NETWORK.

So now let's see what we need to implement in order to make the two apps functional:

        1. Make the connection between the two devices.

        2. On the computer app, create an object which will represent the mobile phone.

        3. Send data from the phone to the computer.

        4. Convert the data received from the phone to a format which can be used to place the object on screen.

        5. Update the position and the orientation of the camera on screen. Display the received values on screen.

Bind the Phone and the Computer

We will have to update both scripts to make sure they communicate with each other.

        1. In UDPSend (phone app), you need to set the IP address of your PC. It's like you want to send an e-mail to your friend. In order to be able to send it, you need to know it's e-mail address. But your friend does not need to know your e-mail address in order to receive your e-mail. So you need to find out, the IP of your computer. For this, do the following steps:

                - Click on the Windows logo.

                - Type Command Promp and open it.

                - In the terminal type ipconfig.

                - Copy the  IPv4 Address from the Wireless LAN adapter Wi-Fi and paste it in your code.

/* TODO 1.1 Set your PC IP */
IP = "192.168.1.4";

        2. Now you need to set a port. You send an e-mail to your friend, but he/she needs to know where the e-mail was sent (maybe he has multiple e-mail addresses such as a Gmail account and a work account). For this we need to decide on which port we send the message to. And we set it in  UDPSend.

/* TODO 1.2 Set a port to send messages to. You can use 1098 */
port = 1098;

        3. We also set the same port in UDPReceive (so the computer knows where the data is received).

/* TODO 1.3 Set a port to listen to (same port from the phone app). You can use 1098 */
port = 1098;

Create a GameObject to represent the Phone

        1. We need to draw something on screen which will be "the mobile phone". For simplicity, we will use a simple cube. But we will update it a little. In order to clearly see the direction the cube is looking at, we will attach the three axis to it.  OX is already attached. Click in Hierarchy on the arrow near the CameraRepresentation game object. Now look how the three axis are created in the CoordinateSystem game object. You can just duplicate the two missing axis from the CoordinateSystem game object and attach them to "the mobile phone". All you need to do now is to rescale and to reposition the newly added axis. Take OX as an example.

Send Data from Phone to Computer

        1. Now we need to send the data from the phone to the computer. In UDPSend,  we can see the rotation is already being sent. You only need to send the position. Notice how we convert anything to a string.

/* TODO 3 Send the camera orientation and position to the tracking app */
sendMessage("ARCore:" + FirstPersonCamera.transform.rotation.ToString() + ";"
                      + FirstPersonCamera.transform.position.ToString());

Process the received Data

        1. The data is received by the computer in UDPReceive. But we cannot use this data directly. We receive it as a string. So you need to convert it from string to numbers again. In order to represent the position we need 3 numbers. But in order to represent a rotation we need 4 numbers. In UDPReceive there is already a function which converts a string to 3 numbers. Now we need to fill the function which converts a string to 4 numbers.

/* Convert string to Vector3 */
public static Vector3 StringToVector3(string sVector)
{
    if (sVector.StartsWith("(") && sVector.EndsWith(")"))
    {
        sVector = sVector.Substring(1, sVector.Length - 2);
    }

    string[] sArray = sVector.Split(',');

    Vector3 result = new Vector3(
        float.Parse(sArray[0]),
        float.Parse(sArray[1]),
        float.Parse(sArray[2]));

    return result;
}

/* TODO 4 Convert string to Quaternion */
public static Quaternion StringToQuaternion(string sVector)
{
    if (sVector.StartsWith("(") && sVector.EndsWith(")"))
    {
        sVector = sVector.Substring(1, sVector.Length - 2);
    }

    string[] sArray = sVector.Split(',');

    Quaternion result = new Quaternion(
        float.Parse(sArray[0]),
        float.Parse(sArray[1]),
        float.Parse(sArray[2]),
        float.Parse(sArray[3]));

    return result;
}

Display the Content on the Screen

Finally, we need to see the data received on screen. Here, you have two steps to do:

        1. Update the position and orientation of the cube which represents the mobile phone.

/* TODO 5.1 Set the camera representation position and rotation to the values received via UDP */
arcore.transform.rotation = orientation;
arcore.transform.position = position;

        2. Display the new values of the position and orientation on screen.

/* TODO 5.2 Display the values received via UDP on screen
* (convert the quaternion to Euler angles for easiness to understand the data)
*/
ARCorePosition.text = "Position: " + position.ToString();
ARCoreRotation.text = "Orientation: " + orientation.ToString();