Getting Started CoreML and Vision – Machine Learning iOS Tutorial


With the launch of xCode 9 Apple has provided support for Machine Learning in CoreML and Vision. This tutorial covers how you can integrate MLModel in an iOS app using Swift 4. We will be making an app in which our app will be able to detect the object it see. For example in the below screen shot this app is able to identify that this is a computer mouse.

Starting new project:

Open xCode and start a new Single View App project. Give it a name and save it.

Now in order to detect an object we need to an ML Model. A machine learning model is a model with already trained data and you can use that data.

Inception v3:

Inception v3 is a model that detects the dominant objects present in an image from a set of 1000 categories such as trees, animals, food, vehicles, people, and more. Goto and download the Inception v3 model.

downloading inception v3
downloading inception v3

After downloading add it to your project just like you add any library and copy item if necessary. After downloading you can see it like this:

Note: If there is no model class in the .mlmodel then you have to add it by going in the Build Phases > Compile Sources.

Designing UI:

The UI Is very simple. Just add a UILabel at the bottom of the view and make it’s IBOutlet. I’ve named it classifier

Let’s Code:

Import these three:

Extend viewController: UIViewController with AVCaptureVideoDataOutputSampleBufferDelegate

In viewDidLoad() add:

In viewDidLoad() we have only made our UIView of ViewController to show the camera preview. If you run now you’ll get and error because you need to add permission in order to access the camera. So in your info.plist add this:

The last thing to do is to add the delegate method captureOutput didOutput. Add this method after viewDidLoad ends

  • //1 we are converting the captured image into PixelBuffer as the model requires that.
  • //2 Here we are initializing the model and calling it.
  • //3 when it is finished analyzing the image it’ll parse the data identifier and confidence. Identifier is the keyword which the model think the object is and confidence is the surety of the identifier like how confident it is about the result.
  • //4 As this identification is automatically happen on background thread so I’ve called the main thread to display what the model has analyzed. Notice I’ve commented some lines these are the lines you want to uncomment if you want to show the value then the model is confident enough about something. You can  change the value from 0.1 to 1 but don’t use big values like 0.7, 0.8 and more because the inception v3 is not that strong.

Here is the full code of the class:

Run the project and you’ll see the result.

Chaudhary Talha

The author Chaudhary Talha

I'm an iOS developer by profession and UI/UX designer by passion. All my tutorials are based on my personal experiences and whenever I learn something new I share it with the world.