Emojify Me

Project Synopsis

Develop a mobile app that takes a picture of people using Firebase Mobile Vision identify all the faces in the photo. Lastly, replace all the faces with the proper emoji.

Goals

• Understand the basics of how artificial intelligence works in particular the field of vision.

• Make use of the Firbase MLKit Vision Api to build a face detector.

• Determine how many faces there are on a photo.

• Identify facial characteristics for each face:

  • Right eye open or close
  • Left eye open or close
  • Smiling or frown

• Replace each face with the proper emoji on top of the original photo.

  • Smile:

  

  • Frown:

  

App overview

This section provides an overview of the code that is provided. The started code allows you to concentrate on the Vision api aspect. However, it is important to understand how this code is architecture and how it works.

Single Activity

The application has a single activity. The activity is the entry point of an Android application. The activity is launched once the user click on the application icon.

Screens

The application defines two screens or fragments. A fragment defines its own UI and encapsulates its own functionality.

Home Fragment

The home fragment defines a bottom app bar with a single FAB button that triggers the camera to take a picture.

Layout

The home fragment is defined on the home_fragment.xml as follows:

• Defines a Coordinator Layout as the main container to respond to scrolling techniques from Toolbars.
• Defines an AppBar to displays information or actions relating to the current screen.
• Defines a Material Tool Bar that provides the activity title and can declare other interactive items.
• Defines a Text View to display the title of the application.
• Defines an include layout with the value of home_content.xml
• Defines a Bottom App Bar to define one main FAB located in the center of the bar.
• Floating Action Button (FAB) to denote the primary action of the screen in this case to take a photo.

The home_content.xml is defined as follows:

• Defines a Constraint Layout that allows placing components according to relationships between sibling views and the parent layout.
• Defines two horizontal guidelines the first one at 35% and the second one at 70% of total height of the screen. These guidelines allow to properly place the Material Card View and the Text View.
• Defines a Material Card View that renders a gif image.
• Defines a Text View that renders the text "Take a Selfie".

HomeFragment.java

This section describes the main aspects of the behavior defined for the home_fragment.xml.

Emojify FAB

Here is the code snippet that is called when the user taps on the emojify FAB:

  @OnClick(R.id.fab_emojify)
  public void onClickEmojifyFAB() {
    // Check for the external storage permission
    // Verify if the app has permission to store a photo on the phone
    if (ContextCompat.checkSelfPermission(getContext(), Manifest.permission.WRITE_EXTERNAL_STORAGE)
        != PackageManager.PERMISSION_GRANTED) {
      // If you do not have permission, request it
      ActivityCompat.requestPermissions(
          getActivity(),
          new String[] {Manifest.permission.WRITE_EXTERNAL_STORAGE},
          REQUEST_STORAGE_PERMISSION);
    } else {
      // Launch the camera if the permission exists
      launchCameraIntent();
    }
  }

The snippet above performs the following actions:

• Verify that the user has permission to store files on the phone.
• In case the user does not have permissions a dialog will be shown asking to grant a permission to store files.
• However, if the user does have permissions then launchCameraIntent() will be invoked.

The goal of method launchCameraIntent() is to launch an intent
that allows the user to take a photo.

launchCameraIntent

  /** Creates a temporary file in which a picture will be store. */
  private void launchCameraIntent() {
    // Create the capture image intent
    Intent takePictureIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
    // Ensure that there's a camera activity to handle the intent
    if (takePictureIntent.resolveActivity(getActivity().getPackageManager()) != null) {
      // Create the temporary File where the photo should go
      File photoFile = null;
      try {
        photoFile = BitmapUtils.createTempImageFile(getContext());
      } catch (IOException ex) {
        // Error occurred while creating the File
        ex.printStackTrace();
      }
      // Continue only if the File was successfully created
      if (photoFile != null) {
        // Get the path of the temporary file
        viewModel.setPhotoPath(photoFile.getAbsolutePath());
        // Get the content URI for the image file
        Uri photoURI = FileProvider.getUriForFile(getContext(), FILE_PROVIDER_AUTHORITY, photoFile);
        // Add the URI so the camera can store the image
        takePictureIntent.putExtra(MediaStore.EXTRA_OUTPUT, photoURI);
        // Launch the camera activity
        startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE);
      }
    }
  }

The snippet above performs the following actions:

• Create an intent that allows the user to take a photo called takePictureIntent.
• Validates that there is an app for taking photos installed on the phone.
• Creates a temporary file in where the photo will be stored photoURI.
• On the intent takePictureIntent specify as an extra where the temporary file on where the photo should be store photoURI.
• Launch the intent startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE).

The call of startActivityForResult() resolves the intent to an app that can handle the intent and starts its corresponding Activity. However, If there is more than one app that can handle the intent, Android presents the user with a dialog to pick which app to use.

onActivityResult

Lastly, once the picture has been taken and temporarily saved. The camera intent will be completed and Android will call the life cycle method onActivityResult:

  @Override
  public void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) {
    super.onActivityResult(requestCode, resultCode, data);
    // If the image capture activity was called and was successful
    if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == Activity.RESULT_OK) {
      // Resample the saved image to fit the ImageView
      Bitmap photo = BitmapUtils.resamplePic(getContext(), viewModel.getPhotoPath());
      // Save the photo on the view model
      viewModel.setPhoto(photo);
      // Navigate to the Photo fragment to see the picture taken
      Navigation.findNavController(getView()).navigate(R.id.action_mainFragment_to_photoFragment);
    } else {
      // Otherwise, delete the temporary image file
      BitmapUtils.deleteImageFile(getContext(), viewModel.getPhotoPath());
    }
  }

The snippet above performs the following actions:

• If intent completed successfully the requestCode will be RESULT_OK. And the temporary file will be stored on the ViewModel. Lastly, the navigation component will render the photo_fragment.xml.
• However, if the intent was not successful the temporary file will be deleted.

Photo Fragment

The photo fragment defines a bottom app bar with a FAB button that saves a picture and two more buttons one to share a photo the second one to delete it.

Layout

The photo fragment is defined on the photo_fragment.xml as follows:

• Defines a Coordinator Layout as the main container to respond to scrolling techniques from Toolbars.
• Defines an AppBar to displays information or actions relating to the current screen.
• Defines a Material Tool Bar that provides the activity title and declares an interactive item to close the current fragment and go back to the home_fragment.xml.
• Defines a Text View to display the title of the application in this case "Photo".
• Defines an include layout with the value of photo_content.xml
• Defines a Bottom App Bar to define one main FAB located in the end of the bar.
• Floating Action Button (FAB) to denote the primary action of the screen in this case to save the current photo on the phone.

The photo_content.xml is defined as follows:

• Defines a Constraint Layout that allows placing components according to relationships between sibling views and the parent layout.
• Defines a horizontal guidelines at 50% of total height of the screen. This guidelines allow to properly place the Material Card View.
• Defines a Material Card View that renders the photo that was take with the camera.

PhotoFragment.java

This section describes the main aspects of the behavior defined for the photo_fragment.xml.

Save FAB

Here is the code snippet that is called when the user taps on the save FAB:

  /** OnClick method for the save button. */
  @OnClick(R.id.fab_save)
  public void onClickSaveFAB() {
    // Save the image
    String photoPath = BitmapUtils.saveImage(getContext(), viewModel.getPhoto());
    viewModel.setPhotoPath(photoPath);
  }

The snippet above performs the following actions:

• Retrieves the photo from the ViewModel and saved it on the phones gallery.
• Upon successfully storing the photo the path is stored on the ViewModel.

Share Action Menu

Here is the code snippet that is called when the user taps on the share action menu:

  /** OnClick method share action menu. */
  private void onShareMenuSelected() {
    onClickSaveFAB();
    // Share the image
    BitmapUtils.shareImage(getContext(), viewModel.getPhotoPath());
  }

The snippet above performs the following actions:

• Calls the onClickSaveFAB to save the photo.
• Lastly, a share intent is trigger to allow the user share a photo through a list of different apps such as Google Photos, Google Drive to name a few.

Delete Action Menu

Here is the code snippet that is called when the user taps on the delete action menu:

  /** OnClick method delete action menu. */
  private void onDeleteMenuSelected() {
    // Delete the temporary image file
    BitmapUtils.deleteImageFile(getContext(), viewModel.getPhotoPath());
  }

The snippet above performs the following actions:

• Deletes the temporary file where the photo is stored.

MaterialToolbar Navigation OnClickListener

Here is the code snippet that is called when the user taps on the Material Tool Bar Navigation:

   // Specifies the close navigation listener to go back to the home fragment
   topToolbar.setNavigationOnClickListener(
      v ->
           Navigation.findNavController(getView())
                .navigate(R.id.action_photoFragment_to_mainFragment));

The snippet above performs the following actions:

• Closes the photo_fragment.xml and navigates back to the home_fragment.xml.

Navigation between fragments

The application make use of navigation component to navigate between the screens. Navigation refers to the interactions that allow users to navigate across, into, and back out from the different screens of an application.

Main Activity Layout

One of the core parts of the Navigation component is the navigation host.
The navigation host is an empty container where destinations are swapped in and out as a user navigates through your app.

<androidx.fragment.app.FragmentContainerView
        android:id="@+id/nav_host_fragment"
        android:name="androidx.navigation.fragment.NavHostFragment"
        android:layout_width="0dp"
        android:layout_height="0dp"
        app:defaultNavHost="true"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintLeft_toLeftOf="parent"
        app:layout_constraintRight_toRightOf="parent"

        app:layout_constraintTop_toTopOf="parent"
        app:navGraph="@navigation/nav_graph" />

Add destinations to the navigation graph

The destinations are defined on the nav_graph.xml as follows:

<?xml version="1.0" encoding="utf-8"?>
<navigation xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:id="@+id/nav_graph"
    app:startDestination="@id/mainFragment">

    <fragment
        android:id="@+id/mainFragment"
        android:name="com.google.codenext.emojify.ui.HomeFragment"
        android:label="main_fragment"
        tools:layout="@layout/home_fragment" >
        <action
            android:id="@+id/action_mainFragment_to_photoFragment"
            app:destination="@id/photoFragment" />
    </fragment>
    <fragment
        android:id="@+id/photoFragment"
        android:name="com.google.codenext.emojify.ui.PhotoFragment"
        android:label="photo_fragment"
        tools:layout="@layout/photo_fragment" >
        <action
            android:id="@+id/action_photoFragment_to_mainFragment"
            app:destination="@id/mainFragment" />
    </fragment>
</navigation>

Navigate to a destination

Navigating to a destination is done using a NavController, an object that manages app navigation within a NavHost. Therefore, to navigate from one destination to another is done via the following method:

Navigation.findNavController(View)

• To Navigate from photo_fragment.xml to home_fragment.xml is accomplished as follow:

Navigation.findNavController(getView())
                .navigate(R.id.action_photoFragment_to_mainFragment));

• To Navigate from home_fragment.xml to photo_fragment.xml is accomplished as follow:

Navigation.findNavController(getView())
                .navigate(R.id.action_mainFragment_to_photoFragment);

ViewModel that stores information

A ViewModel is a class designed to store and manage UI-related data in a lifecycle conscious way. The ViewModel class allows data to survive configuration changes such as screen rotations.

MainActivityViewModel

The ViewModel allows to share the photo and the location where the photo is stored on the phone to allow the fragments communicated among them.

public class MainActivityViewModel extends ViewModel {

  private static final String TAG = "MainActivityViewModel";
  private String photoPath;
  private Bitmap photo;

  /** Provides the file path of the where a photo is stored on the device. */
  public String getPhotoPath() {
    return photoPath;
  }

  /** Stores the file path of where a given photo is stored on the device. */
  public void setPhotoPath(String photoPath) {
    Timber.tag(TAG).d(photoPath);
    this.photoPath = photoPath;
  }

  /** Provides a {@link Bitmap} JPG photo. */
  public Bitmap getPhoto() {
    return photo;
  }

  /** Stores a JPG photo as a {@link Bitmap} */
  public void setPhoto(Bitmap photo) {
    this.photo = photo;
  }
}

Share data between fragments

It's very common that two or more fragments in an activity need to communicate with each other. For our app we need to share the photo taken with the camera as well as its location on where is stored on the phone.

To accomplish this these fragments share a ViewModel using MainActivity scope to handle this communication.

• This is defined in the HomeFragment as follows:

/** Home fragment initiates the camera that captures a selfie in landscape. */
public class HomeFragment extends Fragment {

  private MainActivityViewModel viewModel;
    
  @Override
  public void onCreate(@Nullable Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
   
    viewModel = ViewModelProviders.of(getActivity()).get(MainActivityViewModel.class);
  }
  ...

• This is defined in the PhotoFragment as follows:

/** Photo fragment that allows to emojify a selfie as well as allow to save it, share it or discard it. */
public class PhotoFragment extends Fragment {

  private MainActivityViewModel viewModel;

  @Override
  public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    
    viewModel = ViewModelProviders.of(getActivity()).get(MainActivityViewModel.class);
  }
  ...