Creating your first ARCore application

In the previous article, I already explained what ARCore is and how it helps developers create amazing augmented reality applications without having to understand math and OpenGL.

If you haven’t read it yet, I highly recommend it before turning to this article and start developing ARCore applications.

Beginning of work

To start developing ARCore applications, you first need to add ARCore support to your project. This is very simple, as we will use the Android Studio and Sceneform SDK. There are two basic operations that, thanks to the Sceneform, are performed automatically:

1. Check for ARCore.
2. Request for permission to use the camera.

You do not need to worry about these two steps when creating an ARCore application using the Sceneform SDK. You just need to add the Sceneform SDK to your project.

Create a new Android Studio project with an empty Activity.

Add the following dependency to the project level build.gradle file:

 dependencies { classpath 'com.google.ar.sceneform:plugin:1.5.0' } 

Add this dependency to the application level in the build.gradle file:

 implementation "com.google.ar.sceneform.ux:sceneform-ux:1.5.0" 

Now synchronize the project with the Gradle files and wait for the build to complete. Thus, the Sceneform SDK and the Sceneform plugin for Android Studio will be added to the project. This will allow you to view .sfb , which are 3D models that will be rendered in your camera, as well as help you import, view, and create 3D resources.

Creating your first ARCore application

Now that the Android Studio setup is complete and the Sceneform SDK is installed, we can start creating our first ARCore application.

First, we need to add the Sceneform fragment to our layout. This is the so-called scene where all our 3D models will be placed. The fragment will independently take care of the camera initialization and resolution processing.

Navigate to your main layout file. In my case, this is the activity_main.xml file. And add the Sceneform fragment there:

 <?xml version="1.0" encoding="utf-8"?> <FrameLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools" android:layout_width="match_parent" android:layout_height="match_parent" tools:context=".MainActivity"> <fragment android:name="com.google.ar.sceneform.ux.ArFragment" android:id="@+id/ux_fragment" android:layout_width="match_parent" android:layout_height="match_parent" /> </FrameLayout> 

I set the width and height of match_parent so that the scene match_parent entire screen. You can choose the sizes according to your requirements.

Compatibility check

This is all you need to do in the layout file. Now we turn to Activity, in my case this is MainActivity . Add a method to the Activity:

 public static boolean checkIsSupportedDeviceOrFinish(final Activity activity) { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) { Log.e(TAG, "Sceneform requires Android N or later"); Toast.makeText(activity, "Sceneform requires Android N or later", Toast.LENGTH_LONG).show(); activity.finish(); return false; } String openGlVersionString = ((ActivityManager) activity.getSystemService(Context.ACTIVITY_SERVICE)) .getDeviceConfigurationInfo() .getGlEsVersion(); if (Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) { Log.e(TAG, "Sceneform requires OpenGL ES 3.0 later"); Toast.makeText(activity, "Sceneform requires OpenGL ES 3.0 or later", Toast.LENGTH_LONG) .show(); activity.finish(); return false; } return true; } 

This method checks whether your device supports the Sceneform SDK or not. The SDK requires Android API level 27 or higher and OpenGL ES version 3.0 or higher. If the device does not support these two parameters, the scene will not be loaded, and your application will display a blank screen.

However, you can still implement all the other functions of your application that do not require the Sceneform SDK.

After checking the compatibility, we can create our 3D model and attach it to the scene.

Add assets

Now you need to add to the project a 3D model that will be displayed on your screen. You can create these models yourself if you are familiar with the process of creating them. Or you can go to the Poly .

There you will find a huge repository of 3D resources to choose from. In addition, they are free to download.

In Android Studio, open your application folder in the left pane. You need a sampledata folder. This folder will contain all your 3D models. Inside this folder, create a folder with the name of your model.

In the archive that you download from Poly, you will most likely find 3 files:

1. .mtl file
2. .obj file
3. .png file

The most important of these three files is the .obj file. This is your model. Place all 3 files in sampledata -> “your model folder” .

Now right-click on the .obj file. The first option would be Import Sceneform Asset . Click on it, do not change the default settings, just click Finish in the next window. After that, synchronize the project with the Gradle files.

Import of the 3D resource that will be used in your project is completed. Next, let's use the 3D model in our code and include it in the scene.

Creating a model

Add the following code to your Activity, and I will explain it line by line:

 private static final String TAG = MainActivity.class.getSimpleName(); private static final double MIN_OPENGL_VERSION = 3.0; ArFragment arFragment; ModelRenderable lampPostRenderable; @Override @SuppressWarnings({"AndroidApiChecker", "FutureReturnValueIgnored"}) protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); if (!checkIsSupportedDeviceOrFinish(this)) { return; } setContentView(R.layout.activity_main); arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment); ModelRenderable.builder() .setSource(this, Uri.parse("LampPost.sfb")) .build() .thenAccept(renderable -> lampPostRenderable = renderable) .exceptionally(throwable -> { Toast toast = Toast.makeText(this, "Unable to load andy renderable", Toast.LENGTH_LONG); toast.setGravity(Gravity.CENTER, 0, 0); toast.show(); return null; }); } 

First we find the arFragment that we previously added to the layout. This fragment is responsible for the storage and operation of the scene. You can imagine it as a container for our scene.

Next, we use the ModelRenderable class to build our model. Using the setSource method setSource we load our model from an .sfb file that was generated when importing resources. The thenAccept method gets the model after it is created, and we set the loaded model to our lampPostRenderable variable.

For error handling, we have the exceptionally method, which is called when an exception occurs.

All this happens asynchronously, so you don’t need to worry about multithreading.

Now that the model has been loaded and stored in the lampPostRenderable variable, we will add it to our scene.

Adding a model to the scene

The arFragment is our scene, and it will receive user touch events. Therefore, we need to install an onTap listener for our fragment in order to handle the touches and place objects where it is needed. Add the following code to the onCreate method:

 arFragment.setOnTapArPlaneListener( (HitResult hitresult, Plane plane, MotionEvent motionevent) -> { if (lampPostRenderable == null){ return; } Anchor anchor = hitresult.createAnchor(); AnchorNode anchorNode = new AnchorNode(anchor); anchorNode.setParent(arFragment.getArSceneView().getScene()); TransformableNode lamp = new TransformableNode(arFragment.getTransformationSystem()); lamp.setParent(anchorNode); lamp.setRenderable(lampPostRenderable); lamp.select(); } ); 

We set the onTapArPlaneListener listener for our AR fragment. The following is the lambda expression syntax. If you are not familiar with it, then read this small guide on this topic.

First we create an anchor from HitResult using hitresult.createAnchor() and save it in the Anchor object.

Then we create a knot from this anchor. It will be called AnchorNode and will be attached to the scene using the setParent method.

Next, we create a TransformableNode , which will be our model, and bind it to our node. TransformableNode still does not have any information about the object it should display. We will pass this object to it using the setRenderable method, which takes as an argument an object of type ModelRenderable (remember, we obtained such an object and called it lampPostRenderable ?). And finally, call the method lamp.select() ;

Oh! Too much terminology. Do not worry, now I will explain everything:

1. Scene : this is the place where all your 3D objects will be displayed. This scene is located in the AR fragment that we added to the layout.

   
   

2. HitResult : this is an imaginary line (or ray) coming from infinity, which gives the intersection point of itself with the object of the real world.

   
   

3. Anchor : This is a fixed location and orientation in the real world. It can be understood as coordinates (x, y, z) in three-dimensional space. Pose is the position and orientation of an object on the stage. It is used to transform the local coordinate space of an object into real coordinate space.

   
   

4. Anchor node : this is a node that automatically positions itself in the real world. This is the first node that is installed when a plane is detected.

   
   

5. TransformableNode : This is a node with which to interact. It can be moved, scaled, rotated and so on. In this example, we can scale our object and rotate it. Hence the name Transformable.

   
   

There is no rocket science here. It is really relatively simple. The entire scene can be viewed as a graph, in which the parent object is the scene, and the child - anchor nodes, which then branch out into various other nodes and objects that will be displayed on the screen.

As a result, your Activity should look like this:

 package com.ayusch.arcorefirst; import android.app.Activity; import android.app.ActivityManager; import android.content.Context; import android.net.Uri; import android.os.Build; import android.support.v7.app.AppCompatActivity; import android.os.Bundle; import android.util.Log; import android.view.Gravity; import android.view.MotionEvent; import android.widget.Toast; import com.google.ar.core.Anchor; import com.google.ar.core.HitResult; import com.google.ar.core.Plane; import com.google.ar.sceneform.AnchorNode; import com.google.ar.sceneform.rendering.ModelRenderable; import com.google.ar.sceneform.ux.ArFragment; import com.google.ar.sceneform.ux.TransformableNode; public class MainActivity extends AppCompatActivity { private static final String TAG = MainActivity.class.getSimpleName(); private static final double MIN_OPENGL_VERSION = 3.0; ArFragment arFragment; ModelRenderable lampPostRenderable; @Override @SuppressWarnings({"AndroidApiChecker", "FutureReturnValueIgnored"}) protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); if (!checkIsSupportedDeviceOrFinish(this)) { return; } setContentView(R.layout.activity_main); arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.ux_fragment); ModelRenderable.builder() .setSource(this, Uri.parse("LampPost.sfb")) .build() .thenAccept(renderable -> lampPostRenderable = renderable) .exceptionally(throwable -> { Toast toast = Toast.makeText(this, "Unable to load andy renderable", Toast.LENGTH_LONG); toast.setGravity(Gravity.CENTER, 0, 0); toast.show(); return null; }); arFragment.setOnTapArPlaneListener( (HitResult hitresult, Plane plane, MotionEvent motionevent) -> { if (lampPostRenderable == null){ return; } Anchor anchor = hitresult.createAnchor(); AnchorNode anchorNode = new AnchorNode(anchor); anchorNode.setParent(arFragment.getArSceneView().getScene()); TransformableNode lamp = new TransformableNode(arFragment.getTransformationSystem()); lamp.setParent(anchorNode); lamp.setRenderable(lampPostRenderable); lamp.select(); } ); } public static boolean checkIsSupportedDeviceOrFinish(final Activity activity) { if (Build.VERSION.SDK_INT < Build.VERSION_CODES.N) { Log.e(TAG, "Sceneform requires Android N or later"); Toast.makeText(activity, "Sceneform requires Android N or later", Toast.LENGTH_LONG).show(); activity.finish(); return false; } String openGlVersionString = ((ActivityManager) activity.getSystemService(Context.ACTIVITY_SERVICE)) .getDeviceConfigurationInfo() .getGlEsVersion(); if (Double.parseDouble(openGlVersionString) < MIN_OPENGL_VERSION) { Log.e(TAG, "Sceneform requires OpenGL ES 3.0 later"); Toast.makeText(activity, "Sceneform requires OpenGL ES 3.0 or later", Toast.LENGTH_LONG) .show(); activity.finish(); return false; } return true; } } 

Congratulations! You have just completed creating your first ARCore application. Start adding objects to it and see how they start to come to life in the real world.

This was your first look at how to create a simple ARCore application from scratch in Android Studio. In the next tutorial, I will delve into ARCore and add more functionality to the application.