How to create an OpenCV project for Android
OpenCV is probably the most complete and famous open source library to create awesome computer vision powered applications and projects. In this tutorial I will explain my way to create standalone OpenCV projects for Android, without having to install the OpenCV Manager.
Table of contents
What you will learn
- How to setup the OpenCV library on Android without using the OpenCV Manager.
- How to start using the camera view that comes with the library.
- How to do simple image processing with the captured frames.
Getting the OpenCV Android SDK
The first step to create an OpenCV project in Android is to download the official SDK, which you can get from the official website. Go to the Releases page and click on the latest release for Android (3.4.2 to date) and download it.
Once downloaded, extract the files and place them somewhere safe in your computer. Here, $ANDROID_OPENCV_SDK will be the path where we will install it:
cd ~/Downloads unzip opencv-3.4.2-android-sdk.zip
mkdir -p $ANDROID_OPENCV_SDK
mv OpenCV-android-sdk $ANDROID_OPENCV_SDK
rm opencv-3.4.2-android-sdk.zip
Setting up the project
Create a new Android project
Now that you have the Android OpenCV SDK extracted on a safe location, create a new Android project. For this example I am using Java, targeting the Android API 19 and newer, and only using an empty activity.
Import the Java module
Once the new project is created, we will have to import the Java module from the Android OpenCV SDK. This module includes all the source files necessary to work with OpenCV using Java which, in my opinion, makes it way easier to work with compared with C++.
The easiest way to import this module is by following a wizard located on: File > New > Import Module....
Now in your project you should have a second module next to your app module, called opencv:
The OpenCV module contains the definitions of what we can do with OpenCV, i.e. it contains all the classes, functions and constants needed to use OpenCV, and it should not be modified (unless you really know what you’re doing). Then, we have the app module, where we define how we are going to use the definitions contained by the OpenCV module in our application. To do so, the OpenCV module should become a dependency of the app module.
The way we manage dependencies between modules is through Gradle. In an Android project, each module has a file called build.gradle , where we can define things such as the target version of Android, tasks to run, and of course, the dependencies each module has.
But before we start and add the OpenCV module as a dependency, we have to match the same versions of the app module with the OpenCV module, like this:
android {
compileSdkVersion 27 // same as the app module
buildToolsVersion "27.0.3"
defaultConfig {
minSdkVersion 19 // same as the app module
targetSdkVersion 27 // same as the app module
}
// ...
}
Sync your project, and now you can have access to the OpenCV module from the app module – Awesome! However, something is missing…
Adding the native libraries
If you try to compile and run the project right now, every time you call an OpenCV function the app will crash. That’s because we are missing the OpenCV native libraries.
When you start digging a little deep inside the OpenCV Java module, you will notice that a lot of the code manages calls to native functions, and to actually work we need to have in our project this libraries.
The libraries already come with the Android OpenCV SDK we downloaded earlier, and to use them, we only have to copy them to the app module and then link them to the project. This can be done by executing the next command on the root of the Android project:
cp -r $ANDROID_OPENCV_SDK/sdk/native/libs app/src/main/jniLibs
Now, your app module should have listed a new directory with the libraries from the SDK:
Protip: If you want to save some space on your git repo, and not include the whole jniLibs directory, you can create a symbolic link instead. The only drawback is that other developers will have to have the SDK on the same path:
ln -s $ANDROID_OPENCV_SDK/sdk/native/libs app/src/main/jniLibs
As you may expect, this setup includes every single library that you just added to your final apk, which is bad news for the final user: no one wants to download a 100Mb app! That’s why we’re going to modify the build.gradle of the app module to fix this:
apply plugin: 'com.android.application'
android {
// ...
defaultConfig {
// ...
// add this:
splits {
abi {
enable true
reset()
include 'armeabi-v7a', 'arm64-v8a', 'x86', 'x86_64'
universalApk true
}
}
}
// ...
}
// ...
Now, just sync your project and wait.
Using OpenCV on Android
Statically loading the library
After the last sync, you should be able to compile the application with the OpenCV library. But to before we use its functions, we must first initialize it from Java. Add the following lines to the MainActivity.java file:
public class MainActivity extends AppCompatActivity {
// add this:
static {
System.loadLibrary("opencv_java3");
}
// ...
}
To see if everything is working correctly, we can dump the build information through logcat:
public class MainActivity extends AppCompatActivity {
// ...
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// log it with logcat:
Log.d("MainActivity", "onCreate: OpenCV build information:
+ Core.getBuildInformation());
}
}
Once the app is running, the logs should output something like this:
Using the camera
Now that we know everything is loaded correctly, let’s set up the camera! Go to your MainActivity view, the activity_main.xml file, delete the default TextView and then add the OpenCV camera, called JavaCameraView:
<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">
<!-- add this: -->
<org.opencv.android.JavaCameraView
android:id="@+id/camera_view"
android:layout_width="match_parent"
android:layout_height="match_parent"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintLeft_toLeftOf="parent"
app:layout_constraintRight_toRightOf="parent"
app:layout_constraintTop_toTopOf="parent" />
</android.support.constraint.ConstraintLayout>
And reference it on the MainActivity.java file:
public class MainActivity extends AppCompatActivity {
// add this:
private JavaCameraView cameraView;
// ...
@Override
protected void onCreate(Bundle savedInstanceState) {
// ...
// add this:
cameraView = findViewById(R.id.camera_view);
}
}
Before we run it, we will have to modify the manifest file and allow the app to take control of the camera, just add the following line to your app module’s AndroidManifest.xml:
<uses-permission android:name="android.permission.CAMERA" />
While we’re adding permissions to the manifest, let’s also lock the screen rotation of the Main Activity to “reverse landscape”, so we can fix a little bug with the camera rotation:
<activity android:name=".MainActivity"
android:screenOrientation="reverseLandscape"> <!-- <== add this line -->
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
Since we’re targeting a newer version of Android, don’t forget to ask permission at runtime, you can do it by adding the next line to your onCreate method:
public class MainActivity extends AppCompatActivity {
// add this:
private final int REQUEST_CODE_CAMERA = 1;
// ...
@Override
protected void onCreate(Bundle savedInstanceState) {
// ...
// add this:
ActivityCompat.requestPermissions(MainActivity.this,
new String[]{Manifest.permission.CAMERA},
REQUEST_CODE_CAMERA);
}
}
And then override the onRequestPermissionsResult method to handle the result:
public class MainActivity extends AppCompatActivity {
// ...
// add this:
@Override
public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
switch (requestCode) {
case REQUEST_CODE_CAMERA:
if (grantResults.length > 0
&& grantResults[0] == PackageManager.PERMISSION_GRANTED) {
setupCamera();
}
}
}
}
If the user gives the app access to the camera, a new function, called setupCamera should be called. Which, for now, will only enable the view to start the camera:
public class MainActivity extends AppCompatActivity {
// ...
// add this:
private void setupCamera() {
cameraView.enableView();
}
// ...
}
Processing frames
Once JavaCameraView is activated, we will be able to see the frames that the device’s camera is capturing when we run our app. Which is nice – but now let’s actually process what the camera is seeing!
To process a frame, we must first capture it from a Java interface called CvCameraListener. Implement it on the MainActivity class as following:
public class MainActivity extends AppCompatActivity
implements CameraBridgeViewBase.CvCameraViewListener { // <-- add this
// ...
}
And be sure to implement its methods:
public class MainActivity extends AppCompatActivity
implements CameraBridgeViewBase.CvCameraViewListener {
// ...
@Override
public void onCameraViewStarted(int width, int height) {
}
@Override
public void onCameraViewStopped() {
}
@Override
public Mat onCameraFrame(Mat inputFrame) {
return null;
}
}
As you may have already guessed, onCameraFrame is the method we will use to process the frames. However, it won’t capture any frame until we register a listener, which in this case would be the activity itself since it is implementing the CvCameraListener interface:
public class MainActivity extends AppCompatActivity
implements CameraBridgeViewBase.CvCameraViewListener {
// ...
private void setupCamera() {
cameraView.enableView();
// add this:
cameraView.setCvCameraViewListener(this);
}
// ...
}
To finish, let’s create a simple binary image filter from each frame to see if everything is working correctly:
public class MainActivity extends AppCompatActivity
implements CameraBridgeViewBase.CvCameraViewListener {
// ...
@Override
public Mat onCameraFrame(Mat inputFrame) {
// add this:
Imgproc.cvtColor(inputFrame, inputFrame, Imgproc.COLOR_BGR2GRAY);
Imgproc.threshold(inputFrame, inputFrame, 100, 255, Imgproc.THRESH_BINARY);
return inputFrame;
}
}
The inputFrame object will be modified, and then it will replace the current frame on the camera preview.
Final results
After following this tutorial, you should finish with something like this:
This concludes the tutorial! Thank you so much for reading it, and I hope you learned something from it. You can download this project from here.