Mobile Vision - Face Detection Using On-Device Models
In this chapter, we will build a Flutter application that is capable of detecting faces from media uploaded from the gallery of a device or directly from the camera using the ML Kit's Firebase Vision Face Detection API. The API leverages the power of pre-trained models hosted on Firebase and provides the application, the ability to identify the key features of a face, detect the expression, and get the contours of the detected faces. As the face detection is performed in real time by the API, it can also be used to track faces in a video sequence, in a video chat, or in games that respond to the user's expression. The application, coded in Dart, will work efficiently on Android and iOS devices.
In this chapter, we will be covering the following topics:
- Introduction to image processing
- Developing a...
Technical requirements
You require Visual Studio Code with Flutter and the Dart plugin, and will need to setup the Firebase console. The GitHub repository for this chapter is at https://github.com/PacktPublishing/Mobile-Deep-Learning-Projects/tree/master/Chapter2.
Introduction to image processing
In this chapter, we shall be detecting faces in images. In the context of artificial intelligence, the action of processing an image for the purpose of extracting information about the visual content of that image is called image processing.
Image processing is an emerging field, thanks to the surge in the number of better AI-powered cameras, medical imagery-based machine learning, self-driving vehicles, analysis of people's emotions from images, and many other applications.
Consider the use of image processing by a self-driving vehicle. The vehicle needs to make decisions in as close to real time as possible to ensure the best possible accident-free driving. A delay in the response of the AI model running the car could lead to catastrophic consequences. Several techniques and algorithms have been developed for fast and accurate...
Developing a face detection application using Flutter
With the basic understanding of how a CNN works from Chapter 1, Introduction to Deep Learning for Mobile, and how image processing is done at the most basic level, we are ready to proceed with using the pre-trained models from Firebase ML Kit to detect faces from the given images.
We will be using the Firebase ML Kit Face Detection API to detect the faces in an image. The key features of the Firebase Vision Face Detection API are as follows:
- Recognize and return the coordinates of facial features such as the eyes, ears, cheeks, nose, and mouth of every face detected.
- Get the contours of detected faces and facial features.
- Detect facial expressions, such as whether a person is smiling or has one eye closed.
- Get an identifier for each individual face detected in a video frame. This identifier is consistent across invocations...
Summary
In this chapter, we examined the concept behind image processing and how we can integrate it with our Android- or iOS-based application made using Flutter to perform face detection. The chapter started with adding relevant dependencies to support the functionalities of Firebase ML Kit and the image_picker library. The required UI components with the necessary functionalities were added. The implementation mainly covered image file selection using the Flutter plugin and how images can be processed once they are selected. An example of on-device Face Detector model usage was presented, along with an in-depth discussion of the method by which the implementation was carried out.
In the next chapter, we will be discussing how you can create your own AI-powered chatbot that can double-up as a virtual assistant using the Actions on Google platform.&...