Thursday, April 19, 2012

Face detection for Camera

Long long time ago(2010-05-19), I have a post "Android FaceDetector" about how to use android.media.FaceDetector to identify faces in a Bitmap.

Start from Android 4, API Level 14, the android.hardware.Camera class provide the feature of Face Detection. You can build apps to detect face on camera at live.

2 Face detected

The method startFaceDetection() starts the face detection. This should be called after preview is started. The camera will notify Camera.FaceDetectionListener of the detected faces in the preview frame. The detected faces may be the same as the previous ones. Applications should call stopFaceDetection() to stop the face detection. This method is supported if getMaxNumDetectedFaces() returns a number larger than 0. If the face detection has started, apps should not call this again.

When the face detection is running, setWhiteBalance(String), setFocusAreas(List), and setMeteringAreas(List) have no effect. The camera uses the detected faces to do auto-white balance, auto exposure, and autofocus.


If the apps call autoFocus(AutoFocusCallback), the camera will stop sending face callbacks. The last face callback indicates the areas used to do autofocus. After focus completes, face detection will resume sending face callbacks. If the apps call cancelAutoFocus(), the face callbacks will also resume.

After calling takePicture(Camera.ShutterCallback, Camera.PictureCallback, Camera.PictureCallback) or stopPreview(), and then resuming preview with startPreview(), the apps should call this method again to resume face detection.


It's modify from the exercise "Start Camera auto-focusing, autoFocus()" to add function of face detection.

Modify main.xml to add a SurfaceView for camera preview, and a TextView to display the number of face detected.
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
   android:layout_width="fill_parent"
   android:layout_height="fill_parent"
   android:orientation="vertical" >

   <TextView
       android:layout_width="fill_parent"
       android:layout_height="wrap_content"
       android:text="@string/hello" />
   <TextView
       android:id="@+id/prompt"
       android:layout_width="fill_parent"
       android:layout_height="wrap_content"/>
   <SurfaceView
       android:id="@+id/camerapreview" 
       android:layout_width="fill_parent"
       android:layout_height="wrap_content" />

</LinearLayout>


Create /res/layout/control.xml, it's a view overlap with the preview SurfaceView, to provide a button to take picture.
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
 android:id="@+id/background"
   android:orientation="vertical"
   android:layout_width="fill_parent"
   android:layout_height="fill_parent"
   android:gravity="bottom"
   >
<Button
 android:id="@+id/takepicture" 
 android:layout_width="wrap_content"
 android:layout_height="wrap_content"
 android:text=" * Take Picture "
 android:layout_gravity="right"
 android:layout_margin="10px"
 />
</LinearLayout>


Main code, call camera.startFaceDetection() after camera.startPreview(), and call camera.stopFaceDetection() before camera.stopPreview(), also define our FaceDetectionListener to display the result of number of face detected.
package com.exercise.AndroidCamera;

import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.OutputStream;

import android.app.Activity;
import android.content.ContentValues;
import android.content.pm.ActivityInfo;
import android.graphics.PixelFormat;
import android.hardware.Camera;
import android.hardware.Camera.AutoFocusCallback;
import android.hardware.Camera.Face;
import android.hardware.Camera.FaceDetectionListener;
import android.hardware.Camera.PictureCallback;
import android.hardware.Camera.ShutterCallback;
import android.net.Uri;
import android.os.Bundle;
import android.provider.MediaStore.Images.Media;
import android.view.LayoutInflater;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;
import android.view.ViewGroup.LayoutParams;
import android.widget.Button;
import android.widget.LinearLayout;
import android.widget.TextView;

public class AndroidCamera extends Activity implements SurfaceHolder.Callback{

 Camera camera;
 SurfaceView surfaceView;
 SurfaceHolder surfaceHolder;
 boolean previewing = false;
 LayoutInflater controlInflater = null;
 
 Button buttonTakePicture;
 TextView prompt;
 
 final int RESULT_SAVEIMAGE = 0;
 
   /** Called when the activity is first created. */
   @Override
   public void onCreate(Bundle savedInstanceState) {
       super.onCreate(savedInstanceState);
       setContentView(R.layout.main);
       setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
      
       getWindow().setFormat(PixelFormat.UNKNOWN);
       surfaceView = (SurfaceView)findViewById(R.id.camerapreview);
       surfaceHolder = surfaceView.getHolder();
       surfaceHolder.addCallback(this);
       surfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
      
       controlInflater = LayoutInflater.from(getBaseContext());
       View viewControl = controlInflater.inflate(R.layout.control, null);
       LayoutParams layoutParamsControl
        = new LayoutParams(LayoutParams.FILL_PARENT,
        LayoutParams.FILL_PARENT);
       this.addContentView(viewControl, layoutParamsControl);
      
       buttonTakePicture = (Button)findViewById(R.id.takepicture);
       buttonTakePicture.setOnClickListener(new Button.OnClickListener(){

   @Override
   public void onClick(View arg0) {
    // TODO Auto-generated method stub
    camera.takePicture(myShutterCallback,
      myPictureCallback_RAW, myPictureCallback_JPG);
   }});
      
       LinearLayout layoutBackground = (LinearLayout)findViewById(R.id.background);
       layoutBackground.setOnClickListener(new LinearLayout.OnClickListener(){

   @Override
   public void onClick(View arg0) {
    // TODO Auto-generated method stub

    buttonTakePicture.setEnabled(false);
    camera.autoFocus(myAutoFocusCallback);
   }});
      
       prompt = (TextView)findViewById(R.id.prompt);
   }
  
   FaceDetectionListener faceDetectionListener
   = new FaceDetectionListener(){

  @Override
  public void onFaceDetection(Face[] faces, Camera camera) {
   
   if (faces.length == 0){
    prompt.setText(" No Face Detected! ");
   }else{
    prompt.setText(String.valueOf(faces.length) + " Face Detected :) ");
   }
   
   
  }};
  
   AutoFocusCallback myAutoFocusCallback = new AutoFocusCallback(){

  @Override
  public void onAutoFocus(boolean arg0, Camera arg1) {
   // TODO Auto-generated method stub
   buttonTakePicture.setEnabled(true);
  }};
  
   ShutterCallback myShutterCallback = new ShutterCallback(){

  @Override
  public void onShutter() {
   // TODO Auto-generated method stub
   
  }};
  
 PictureCallback myPictureCallback_RAW = new PictureCallback(){

  @Override
  public void onPictureTaken(byte[] arg0, Camera arg1) {
   // TODO Auto-generated method stub
   
  }};
  
 PictureCallback myPictureCallback_JPG = new PictureCallback(){

  @Override
  public void onPictureTaken(byte[] arg0, Camera arg1) {
   // TODO Auto-generated method stub
   /*Bitmap bitmapPicture
    = BitmapFactory.decodeByteArray(arg0, 0, arg0.length); */
   
   Uri uriTarget = getContentResolver().insert(Media.EXTERNAL_CONTENT_URI, new ContentValues());

   OutputStream imageFileOS;
   try {
    imageFileOS = getContentResolver().openOutputStream(uriTarget);
    imageFileOS.write(arg0);
    imageFileOS.flush();
    imageFileOS.close();
    
    prompt.setText("Image saved: " + uriTarget.toString());
    
   } catch (FileNotFoundException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
   } catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
   }

   
   camera.startPreview();
   camera.startFaceDetection();
  }};

 @Override
 public void surfaceChanged(SurfaceHolder holder, int format, int width,
   int height) {
  // TODO Auto-generated method stub
  if(previewing){
   camera.stopFaceDetection();
   camera.stopPreview();
   previewing = false;
  }
  
  if (camera != null){
   try {
    camera.setPreviewDisplay(surfaceHolder);
    camera.startPreview();

    prompt.setText(String.valueOf(
      "Max Face: " + camera.getParameters().getMaxNumDetectedFaces()));
    camera.startFaceDetection();
    previewing = true;
   } catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
   }
  }
 }

 @Override
 public void surfaceCreated(SurfaceHolder holder) {
  // TODO Auto-generated method stub
  camera = Camera.open();
  camera.setFaceDetectionListener(faceDetectionListener);
 }

 @Override
 public void surfaceDestroyed(SurfaceHolder holder) {
  // TODO Auto-generated method stub
  camera.stopFaceDetection();
  camera.stopPreview();
  camera.release();
  camera = null;
  previewing = false;
 }
}


Modify AndroidManifest.xml, to add permission of "android.permission.CAMERA", and set android:minSdkVersion="14".
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
   package="com.exercise.AndroidCamera"
   android:versionCode="1"
   android:versionName="1.0" >

   <uses-sdk android:minSdkVersion="14" />
   <uses-permission android:name="android.permission.CAMERA"/>

   <application
       android:icon="@drawable/ic_launcher"
       android:label="@string/app_name" >
       <activity
           android:name=".AndroidCamera"
           android:label="@string/app_name" >
           <intent-filter>
               <action android:name="android.intent.action.MAIN" />

               <category android:name="android.intent.category.LAUNCHER" />
           </intent-filter>
       </activity>
   </application>

</manifest>


Download the files.

Please note, it's a in-complete exercise. It simple display the number of face detected.

Next: - Android 4 Face Detection: Display detected face area

13 comments:

Viktor Brešan said...

I am running Android 4.1 emulator using webcam as a camera and it returns 0 for getMaxNumDetectedFaces(). Do you know how to set it up so that I can use face detection on a web cam?

Erik said...

getMaxNumDetectedFaces() return 0 means face detection of the specified type is not supported.

http://developer.android.com/reference/android/hardware/Camera.Parameters.html#getMaxNumDetectedFaces()

yudhaardiansyah said...

sorry for distrubing you, but i just want to share the article about face detection,
let's check this link http://repository.gunadarma.ac.id/bitstream/123456789/3365/1/Automed%20Face%20Detection%20and%20Feature%20Extraction%20Using%20Color%20Feret%20Image%20Database.pdf
i wish that article can be usefull

Anonymous said...

Thanks for the example its wonderful but the issue i am getting is that, i am using nexus7 tab with android 4.2, api 17 which supports facedetection. When i run this code i get an fatal exception as invalid facedetecction type = 0,
Can you help me with this.
Thank you in advance.

Erik said...

As my understanding, you have to call getMaxNumDetectedFaces() to check if face detection is supported. if return 0 means face detection of the specified type is not supported. it's hardware related.

But I don't think Nexus 7 not support face detection. Do you run on real Nexus 7? or emulator?

Anonymous said...

Its a real Nexus 7 device

Logan said...

I had tried many ways for using the Face Detection of Level 14 API.I have tried in more than 3 devices.. All the devices shows some value greater than 0 for getMaxNumDetectedFaces(). But the Face detection listener is not at all working... Anyone kindly help me.. I have been struck with this months together.

Anonymous said...

same here, i'm using sIII with Android 4.1.2
can run, but get an error when capturing the camera
(maxNumFace = 0)

Unknown said...

What devices have people got this working on? i have an original nexus 7 and it has 0 for maxdetected faces. has anyone tried this with the new nexus 7?

Anonymous said...

will this work even if the image uploaded is of any other object than face

Anonymous said...

Can we add zoom to the camera in this project

Naveen Kant Mishra said...

A big issue is if you first request for focus and then try to detect face it wont be work..

Anonymous said...

thank you you so much sir for this wonderful project great work....and i want to ask you how to row circle or rectangle on face when face detected.