01d - Face Analysis.ipynb

{"cells":[{"cell_type":"markdown","source":["# Detecting and Analyzing Faces\n","\n","Computer vision solutions often require an artificial intelligence (AI) solution to be able to detect, analyze, or identify human faces. or example, suppose the retail company Northwind Traders has decided to implement a \"smart store\", in which AI services monitor the store to identify customers requiring assistance, and direct employees to help them. One way to accomplish this is to perform facial detection and analysis - in other words, determine if there are any faces in the images, and if so analyze their features.\n","\n","![A robot analyzing a face](./images/face_analysis.jpg)\n","\n","## Use the Face cognitive service to detect faces\n","\n","Suppose the smart store system that Northwind Traders wants to create needs to be able to detect customers and analyze their facial features. In Microsoft Azure, you can use **Face**, part of Azure Cognitive Services to do this.\n","\n","### Create a Cognitive Services Resource\n","\n","Let's start by creating a **Cognitive Services** resource in your Azure subscription.\n","\n","> **Note**: If you already have a Cognitive Services resource, just open its **Quick start** page in the Azure portal and copy its key and endpoint to the cell below. Otherwise, follow the steps below to create one.\n","\n","1. In another browser tab, open the Azure portal at https://portal.azure.com, signing in with your Microsoft account.\n","2. Click the **&#65291;Create a resource** button, search for *Cognitive Services*, and create a **Cognitive Services** resource with the following settings:\n","    - **Name**: *Enter a unique name*.\n","    - **Subscription**: *Your Azure subscription*.\n","    - **Location**: *Choose any available region*:\n","    - **Pricing tier**: S0\n","    - **Resource group**: *Create a resource group with a unique name*.\n","3. Wait for deployment to complete. Then go to your cognitive services resource, and on the **Overview** page, click the link to manage the keys for the service. You will need the endpoint and keys to connect to your cognitive services resource from client applications.\n","\n","### Get the Key and Endpoint for your Cognitive Services resource\n","\n","To use your cognitive services resource, client applications need its endpoint and authentication key:\n","\n","1. In the Azure portal, on the **Keys and Endpoint** page for your cognitive service resource, copy the **Key1** for your resource and paste it in the code below, replacing **YOUR_COG_KEY**.\n","2. Copy the **endpoint** for your resource and and paste it in the code below, replacing **YOUR_COG_ENDPOINT**.\n","3. Run the code in the cell below by clicking the Run Cell <span>&#9655</span> button (at the top left of the cell)."],"metadata":{}},{"cell_type":"code","source":["cog_key = 'YOUR_COG_KEY'\n","cog_endpoint = 'YOUR_COG_ENDPOINT'\n","\n","print('Ready to use cognitive services at {} using key {}'.format(cog_endpoint, cog_key))"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693964655}}},{"cell_type":"markdown","source":["To use the Face service in your Cognitive Services resource, you'll need to install the Azure Cognitive Services Face package. "],"metadata":{}},{"cell_type":"code","source":["! pip install azure-cognitiveservices-vision-face"],"outputs":[],"execution_count":null,"metadata":{"tags":[]}},{"cell_type":"markdown","source":["Now that you have a Cognitive Services resource and the SDK package installed, you can use the Face service to detect human faces in the store.\n","\n","Run the code cell below to see an example."],"metadata":{}},{"cell_type":"code","source":["from azure.cognitiveservices.vision.face import FaceClient\n","from msrest.authentication import CognitiveServicesCredentials\n","from python_code import faces\n","import os\n","%matplotlib inline\n","\n","# Create a face detection client.\n","face_client = FaceClient(cog_endpoint, CognitiveServicesCredentials(cog_key))\n","\n","# Open an image\n","image_path = os.path.join('data', 'face', 'store_cam2.jpg')\n","image_stream = open(image_path, \"rb\")\n","\n","# Detect faces\n","detected_faces = face_client.face.detect_with_stream(image=image_stream)\n","\n","# Display the faces (code in python_code/faces.py)\n","faces.show_faces(image_path, detected_faces)"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693970079}}},{"cell_type":"markdown","source":["Each detected face is assigned a unique ID, so your application can identify each individual face that was detected.\n","\n","Run the cell below to see the IDs for some more shopper faces."],"metadata":{}},{"cell_type":"code","source":["# Open an image\n","image_path = os.path.join('data', 'face', 'store_cam3.jpg')\n","image_stream = open(image_path, \"rb\")\n","\n","# Detect faces\n","detected_faces = face_client.face.detect_with_stream(image=image_stream)\n","\n","# Display the faces (code in python_code/faces.py)\n","faces.show_faces(image_path, detected_faces, show_id=True)"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693970447}}},{"cell_type":"markdown","source":["## Analyze facial attributes\n","\n","Face can do much more than simply detect faces. It can also analyze facial features and expressions to suggest age and emotional state; For example, run the code below to analyze the facial attributes of a shopper."],"metadata":{}},{"cell_type":"code","source":["# Open an image\n","image_path = os.path.join('data', 'face', 'store_cam1.jpg')\n","image_stream = open(image_path, \"rb\")\n","\n","# Detect faces and specified facial attributes\n","attributes = ['age', 'emotion']\n","detected_faces = face_client.face.detect_with_stream(image=image_stream, return_face_attributes=attributes)\n","\n","# Display the faces and attributes (code in python_code/faces.py)\n","faces.show_face_attributes(image_path, detected_faces)"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693971321}}},{"cell_type":"markdown","source":["Based on the emotion scores detected for the customer in the image, the customer seems pretty happy with the shopping experience.\n","\n","## Find similar faces \n","\n","The face IDs that are created for each detected face are used to individually identify face detections. You can use these IDs to compare a detected face to previously detected faces and find faces with similar features.\n","\n","For example, run the cell below to compare the shopper in one image with shoppers in another, and find a matching face."],"metadata":{}},{"cell_type":"code","source":["# Get the ID of the first face in image 1\n","image_1_path = os.path.join('data', 'face', 'store_cam3.jpg')\n","image_1_stream = open(image_1_path, \"rb\")\n","image_1_faces = face_client.face.detect_with_stream(image=image_1_stream)\n","face_1 = image_1_faces[0]\n","\n","# Get the face IDs in a second image\n","image_2_path = os.path.join('data', 'face', 'store_cam2.jpg')\n","image_2_stream = open(image_2_path, \"rb\")\n","image_2_faces = face_client.face.detect_with_stream(image=image_2_stream)\n","image_2_face_ids = list(map(lambda face: face.face_id, image_2_faces))\n","\n","# Find faces in image 2 that are similar to the one in image 1\n","similar_faces = face_client.face.find_similar(face_id=face_1.face_id, face_ids=image_2_face_ids)\n","\n","# Show the face in image 1, and similar faces in image 2(code in python_code/face.py)\n","faces.show_similar_faces(image_1_path, face_1, image_2_path, image_2_faces, similar_faces)"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693972555}}},{"cell_type":"markdown","source":["## Recognize faces\n","\n","So far you've seen that Face can detect faces and facial features, and can identify two faces that are similar to one another. You can take things a step further by inplementing a *facial recognition* solution in which you train Face to recognize a specific person's face. This can be useful in a variety of scenarios, such as automatically tagging photographs of friends in a social media application, or using facial recognition as part of a biometric identity verification system.\n","\n","To see how this works, let's suppose the Northwind Traders company wants to use facial recognition to ensure that only authorized employees in the IT department can access secure systems.\n","\n","We'll start by creating a *person group* to represent the authorized employees."],"metadata":{}},{"cell_type":"code","source":["group_id = 'employee_group_id'\n","try:\n","    # Delete group if it already exists\n","    face_client.person_group.delete(group_id)\n","except Exception as ex:\n","    print(ex.message)\n","finally:\n","    face_client.person_group.create(group_id, 'employees')\n","    print ('Group created!')"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693973492}}},{"cell_type":"markdown","source":["Now that the *person group* exists, we can add a *person* for each employee we want to include in the group, and then register multiple photographs of each person so that Face can learn the distinct facial characetristics of each person. Ideally, the images should show the same person in different poses and with different facial expressions.\n","\n","We'll add a single employee called Wendell, and register three photographs of the employee."],"metadata":{}},{"cell_type":"code","source":["import matplotlib.pyplot as plt\n","from PIL import Image\n","import os\n","%matplotlib inline\n","\n","# Add a person (Wendell) to the group\n","wendell = face_client.person_group_person.create(group_id, 'Wendell')\n","\n","# Get photo's of Wendell\n","folder = os.path.join('data', 'face', 'wendell')\n","wendell_pics = os.listdir(folder)\n","\n","# Register the photos\n","i = 0\n","fig = plt.figure(figsize=(8, 8))\n","for pic in wendell_pics:\n","    # Add each photo to person in person group\n","    img_path = os.path.join(folder, pic)\n","    img_stream = open(img_path, \"rb\")\n","    face_client.person_group_person.add_face_from_stream(group_id, wendell.person_id, img_stream)\n","\n","    # Display each image\n","    img = Image.open(img_path)\n","    i +=1\n","    a=fig.add_subplot(1,len(wendell_pics), i)\n","    a.axis('off')\n","    imgplot = plt.imshow(img)\n","plt.show()"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693976898}}},{"cell_type":"markdown","source":["With the person added, and photographs registered, we can now train Face to recognize each person."],"metadata":{}},{"cell_type":"code","source":["face_client.person_group.train(group_id)\n","print('Trained!')"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693977046}}},{"cell_type":"markdown","source":["Now, with the model trained, you can use it to identify recognized faces in an image."],"metadata":{}},{"cell_type":"code","source":["# Get the face IDs in a second image\n","image_path = os.path.join('data', 'face', 'employees.jpg')\n","image_stream = open(image_path, \"rb\")\n","image_faces = face_client.face.detect_with_stream(image=image_stream)\n","image_face_ids = list(map(lambda face: face.face_id, image_faces))\n","\n","# Get recognized face names\n","face_names = {}\n","recognized_faces = face_client.face.identify(image_face_ids, group_id)\n","for face in recognized_faces:\n","    person_name = face_client.person_group_person.get(group_id, face.candidates[0].person_id).name\n","    face_names[face.face_id] = person_name\n","\n","# show recognized faces\n","faces.show_recognized_faces(image_path, image_faces, face_names)\n","\n"],"outputs":[],"execution_count":null,"metadata":{"gather":{"logged":1599693994820}}},{"cell_type":"markdown","source":["## Learn More\n","\n","To learn more about the Face cognitive service, see the [Face documentation](https://docs.microsoft.com/azure/cognitive-services/face/)\n"],"metadata":{}}],"metadata":{"language_info":{"name":"python","version":"3.6.9","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kernelspec":{"name":"python3-azureml","language":"python","display_name":"Python 3.6 - AzureML"},"kernel_info":{"name":"python3-azureml"},"nteract":{"version":"nteract-front-end@1.0.0"}},"nbformat":4,"nbformat_minor":2}