DEV Community: Nex Mobility

Brief about Face Detection using Vision framework

Nex Mobility — Tue, 23 Jul 2019 06:45:29 +0000

Apple introduced the new vision library used to which is used for various purposes. One of the best features is used to detect face with the landmark of the faces. It can recognize the eyes, brows, ear, nose, and mouth. Using this library developer can crop the detected face, and even a developer can maintain the different condition where it will crop only faces from the whole body.

Even this library can detect the mustaches, glass, and hat. For face detection, the only landmark is not enough a developer might need to create some model to train the model using CoreML to make it advance features. Where developer needs to enroll for CoreML model, upload and train the model for face detection If more the face images uploaded in the cloud then, more will be accuracy. In the vision, the most important is Request, Request Handler, and the Result of the request.

Face detection is not an easy task. There are only a few libraries available in the market to detect face but integrating, with iOS is a big challenge for the developers. So far, the most useful component in the market is OpenCV, but there is limited space, and it is difficult to integrate into iOS app development. During face, crop developer should take a minimum of 20 images at a time to train the model more accurately then zip it and upload in the cloud server. Once the model trained, then the developer needs to capture the uploaded pic to match with a particular user.

Facial recognition is a big challenge and more tedious task for the developer to detect the face and get the 90-100% accuracy, but once it developed then, it can be leveraged in various industry, collages, hotel, and company, etc. This solution is more efficient, and it will minimize the long queue and optimize the system. Even it will make an easier life in many places.

Implementation for Face Detection:

Step 1:
A developer needs to create the request to detect face images using, VNDetectFaceRectanglesRequest then developer need to create the request for VNImageRequestHandler where the developer needs to take care of image type to pass on it.


var facehandlerRequest: VNDetectFaceRectanglesRequest = {
 let facedetect = VNDetectFaceRectanglesRequest(completionHandler: self.facehandlerRequest)
 return facedetect
}()
var facehandlerrequest = VNImageRequestHandler(cgImage: cgImage, options: [: ])
do {
 try handler.perform([self.facehandlerRequest])
} catch {
 print("server error.\n\(error.localizedDescription)")
}

func FaceDetectionHandler(request: VNRequest, error: Error ? ) {
 guard
 let observations = request.results as ? [VNFaceObservation]
 else {
  print("unexpected result type from VNFaceObservation")
  return
 }
 DispatchQueue.main.async {
  for face in observations {
   let view = self.createBoxView(withColor: UIColor.red)
   view.frame = self.transformRect(fromRect: face.boundingBox, toViewRect: self.yourImageView)
   self.yourImageView.addSubview(view)
  }
 }
}

Step 2:
A developer needs to get the array of observations from VNFaceobservation and then need to draw a square/rectangle shape on faces to crop it.


func facelandamrktransformRect(fromRect: CGRect , toViewRect :UIView) -> CGRect {
    var RectDetect = CGRect()
  RectDetect.size.width = fromRect.size.width * toViewRect.frame.size.width
    RectDetect.size.height = fromRect.size.height * toViewRect.frame.size.height
    RectDetect.origin.y =  (toViewRect.frame.height) - (toViewRect.frame.height * fromRect.origin.y )
    RectDetect.origin.y  = toRect.origin.y -  toRect.size.height
    RectDetect.origin.x =  fromRect.origin.x * toViewRect.frame.size.width
    return RectDetect  
}

func createBoxView(withColor: UIColor) - > UIView {
 let view = UIView()
 view.layer.borderColor = withColor.cgColor
 view.layer.borderWidth = 2
 view.backgroundColor = UIColor.clear
 return view
}

Step 3:
After face detection, a developer needs to crop the face from the detected face then only face will be visible and other parts of the body get eliminated from the image’s using the logic, and it can be handled using the code.


public enum FaceCropResultSet {
  case success([T])
  case notFound
  case failure(Error)
}

public struct FaceDetectCropper {
  let detectable: T
  init(_ detectable: T) {
    self.detectable = detectable
  }
}
public protocol FaceCroppable {}
public extension FaceCroppable {
 var face: FaceDetectCropper < Self > {
  return FaceDetectCropper(self)
 }
}
public extension FaceDetectCropper where T: CGImage {
  func crop(_ completion: @escaping(FaceCropResultSet < CGImage > ) - > Void) {
    guard# available(iOS 11.0, *)
    else {
     return
    }
    let req = VNDetectFaceRectanglesRequest { request, error in
      guard error == nil else {
        completion(.failure(error!))
        return
      }
       
       let faceImages = request.results?.map({ result -> CGImage? in
        guard let face = result as? VNFaceObservation else { return nil }
        
        let context = UIGraphicsGetCurrentContext()
        
        // draw the image
        context?.scaleBy(x: 1.0, y: -1.0)
        
        let width = face.boundingBox.width * CGFloat(self.detectable.width)
        let height = face.boundingBox.height * CGFloat(self.detectable.height)
        let x = face.boundingBox.origin.x * CGFloat(self.detectable.width)
        let y = (1 - face.boundingBox.origin.y) * CGFloat(self.detectable.height) - height
        
        let croppingRect = CGRect(x: x, y: y, width: width, height: height)
        let faceImage = self.detectable.cropping(to: croppingRect)
              
        context?.saveGState()
        context?.setStrokeColor(UIColor.black.cgColor)
        context?.setLineWidth(8.0)
        context?.addRect(croppingRect)
        context?.drawPath(using: .stroke)
        context?.restoreGState()
        
        return faceImage
      }).flatMap { $0 }
      
      guard let result = faceImages, result.count > 0 else {
        completion(.notFound)
        return
      }
      
      completion(.success(result))
    }
    
    do {
      try VNImageRequestHandler(cgImage: self.detectable, options: [:]).perform([req])
    } catch let error {
      completion(.failure(error))
    }
  }
}

public extension FaceDetectCropper where T: UIImage {
 func crop(_ completion: @escaping(FaceCropResultSet < UIImage > ) - > Void) {
  guard# available(iOS 11.0, *)
  else {
   return
  }
  self.detectable.cgImage!.face.crop {
   result in
    switch result {
     case.success(let cgFaces):
      let faces = cgFaces.map {
       cgFace - > UIImage in
        return UIImage(cgImage: cgFace)
      }
     completion(.success(faces))
     case.notFound:
      completion(.notFound)
     case.failure(let error):
      completion(.failure(error))
    }
  }
 }
}
extension NSObject: FaceCroppable {}
extension CGImage: FaceCroppable {}

Step 4:
The developer needs to create the custom camera where he can take minimum 20 pics, crop faces and zip it. Once cropping is done then pics can be uploaded in the cloud for CoreML training. Once a model trained with crop face images, then it can be compared in the next step by calling the CoreML API and pass the image in CoreML API. If successful, the face is found, if not found, the message will appear and retry the option will be CoreML.


func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey: Any]) {
 let image = info[UIImagePickerController.InfoKey.originalImage]
 let image1 = info[UIImagePickerController.InfoKey.originalImage] as!UIImage
 image1.face.crop {
  result in
  switch result {
    case.success(let faces):
     for i in 0.. < faces.count {
      self.capturedImages.append(faces[i])
     }
    print("capturedImages", self.capturedImages)
    print("capturedImages", faces.count)
    let string1: String = TrainModelViewController.userID!
     if self.capturedImages.count == 10 {
     self.cameraTimer.invalidate()
     for i in 0.. < self.capturedImages.count {
      let documentsPath1 = NSURL(fileURLWithPath: NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0])
      print("documentsPath1", documentsPath1)
      let folderPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first!+"/" + string1 + "/"
      try ? FileManager.default.createDirectory(at: URL(fileURLWithPath: folderPath), withIntermediateDirectories: false, attributes: nil)
      let filePath = folderPath + "\(String(describing:i))" + ".png"
      print("filepath", filePath)
      let imageData = (image as!UIImage).jpegData(compressionQuality: 0.5)
      FileManager.default.createFile(atPath: filePath, contents: imageData, attributes: nil)

      if try !FileManager.default.contentsOfDirectory(atPath: folderPath).count == self.capturedImages.count {
       do {
        let zipFilePath =
         try Zip.quickZipFiles([URL(fileURLWithPath: folderPath)], fileName: string1)
        print("zip file path", zipFilePath)
        ZipImageViewController.zipfilepath1 = zipFilePath as URL

       } catch {
        print("Something went wrong")
       }
      }
     }
    }
    break
    case.notFound:
     let alert = UIAlertController(title: "Face not detected", message: "Please choose another image", preferredStyle: UIAlertController.Style.alert)
    alert.addAction(UIAlertAction(title: "Ok", style: UIAlertAction.Style.default, handler: nil))
    self.present(alert, animated: true, completion: nil)
    break
    case.failure(let error):
     let alert = UIAlertController(title: "Face not detected", message: "Please choose another image", preferredStyle: UIAlertController.Style.alert)
    alert.addAction(UIAlertAction(title: "Ok", style: UIAlertAction.Style.default, handler: nil))
    self.present(alert, animated: true, completion: nil)
    break
   }
 }
 if !cameraTimer.isValid {
  finishAndUpdate()
 }
}

Step 5:
Sample image: Once image get detected then Cropping will be done by red rectangular area

Enterprise Mobility Strategy – Define the Challenges & Solutions

Nex Mobility — Mon, 24 Jun 2019 06:06:16 +0000

Mobility is growing rapidly and there’s simply no doubt about this fact. Even when it comes to the corporate world, companies have started relying on the power of mobility. Whether it is the use of mobile apps to grow their business or whether the use of mobile applications (Enterprise Mobility Solutions) to improve the performance of the operations, companies are leaving no stone unturned to make the full use of mobility solutions. Now, when it comes to Enterprise Mobility Solutions, they have been into existence since quite a few years, but now, as mobility is growing immensely, thus the industry of Enterprise Mobility Solutions is also expanding. In fact, Enterprise Mobility Solutions and Bring Your Own Device industry is expected to touch the $73.30 billion mark by 2021.

Enterprise Mobility Solutions are specially curated for the corporate!

We are a part of the digital generation and the number of mobile phone users is growing exponentially. Thus, the corporate world has to also succumb to the power of mobility in order to succeed in this era. One way to make use of the mobility solutions is via mobile apps which are developed for the customers. That’s not it. Mobility solutions are also implemented in the workplaces in the form of ‘Enterprise Mobility Solutions’. And, this is the time where such solutions are preferred more than ever as the use and importance of mobiles is constantly increasing.

Why are companies adopting Enterprise Mobility Solutions more than ever?

One of the major factors that contributed to the growth of Enterprise Mobility Solutions is the increasing use of mobiles. Almost every employee of the company is using mobile phones. In fact, a lot of times we use mobiles for replying important emails or conversing about important office work. Therefore, there’s nothing better than making it possible for the employees to handle a lot of their world on mobiles. And, this can only happen if the companies adopt latest Enterprise Mobility Solutions.

Many firms have in fact stated adopting Enterprise Mobility Solutions to empower their employees. As, Enterprise Mobility Solutions not only make the work convenient for the employees. But, they also help the employees to boost the speed and efficiency of their operations. Therefore, more and more businesses these days are adopting the new-age Enterprise Mobility Solutions.

Enterprise Mobility Solutions are constantly evolving

The demand of the Enterprise Mobility Solutions is increasing. And, the developers are trying to upgrade the solutions to make them perfect for the current generation of users. A plenty of new technologies have been introduced in the recent times and a lot of these technologies are being implement in various new Enterprise Mobility Solutions as well. Also, the Enterprise Mobility Solutions are improved based on the feedback from the users as well. However, still, there are a few challenges that the businesses and the developers of Enterprise Mobility Solutions are trying to tackle.

Challenges in the world of Enterprise Mobility Solutions

Though, the world of Enterprise Mobility Solutions seems to be pretty bright but there are some obstacles that are creating a bit of issues for the users and the developers. Some of the challenges faced by the Enterprise Mobility Solutions’ industry and the users are:

Traditional Systems - One of the key challenges faced in the Enterprise Mobility Solutions’ industry is the legacy systems. Many organizations find it difficult to manage and use the legacy frameworks for the modern Enterprise Mobility Solutions. Also, the use of legacy systems makes the coordination to and from different groups extremely tough. Many big sized firms are so accustomed to using their legacy systems that they don’t want to give up. But, of they have to adopt the latest Enterprise Mobility Solutions, then it is important to let go of the traditional systems and to adopt cutting-edge systems and programs.

Security - Security is definitely one of the key challenges faced by any industry that is linked to the use of cyberspace; and the connection of devices. One of the biggest hurdles faced by the Enterprise Mobility Solutions’ industry and the Bring Your Own Devices culture is the security of the company’s data including customers’ and clients’ confidential information. Increased use of Enterprise Mobility Solutions needs a very powerful security system which safeguards the data of the company. The increasing stress relating to cyber security is making it all the more important for the firms to first strengthen their systems and then adopt the Enterprise Mobility Solutions.

Fragmentation - Businesses, especially the ones who implement the BYOD policy are flooded with the high number of connected devices. The increasing volume of diversity in mobility is creating a situation of fragmentation. Though, the users of the Enterprise Mobility Solutions want to experience a hassle free experience across different devices, but it can only be possible with a great level of symmetry. Thus, the Enterprise Mobility Solutions used by the firms should be tested on a large variety of different devices, to make sure that the users get a similar type of experience across different platforms.

Budget is also one of the major concerns faced by the companies when it comes to adopting Enterprise Mobility Solutions, as there is a certain amount of cost that comes with it. But, firms have to realize that the benefits of using Enterprise Mobility Solutions are enormous. It’s just that firms have to be prepared to cross the hurdles to make the most of Enterprise Mobility Solutions.

5 iOS 12 bugs that are the bane of the life of the Apple users!

Nex Mobility — Wed, 17 Oct 2018 06:42:27 +0000

iOS 12 is the latest buzzword in the world of Apple. The new operating system is known to be the most stunning till date. Not only has it improved the performance of the iPhones, but it has added a lot of features to the phones as well. This feature-rich edition of the OS is known to bring several interesting features to the phone. It all seems to be beneficial in terms of the new update, but are there any chinks in the armor?

Yes, indeed. There are a plenty of benefits of updating the phone to iOS 12. Therefore, more than 50% of users have already started using iOS 12. But, at the same time, there are some bugs which might hinder the performance of the user as well. Though iOS 12 seems to be a very promising OS and there was a lot of buzz surrounding its inception as well, but now, Apple users are all the more excited to use it when it is finally out. However, some of the irksome issues in iOS 12 are making it difficult for the users to enjoy the update to the fullest. Expert iPhone App Developers India listed below are some of the most troublesome bugs:

1. Problem with the colors
Some of the Apple users have reported that after upgrading iPhone X to iOS 12, the colors looked faded. Though, people think that probably because Apple has tweaked the display profile (iPhone X’s OLED screen) this has happened. However, the real reasons behind this issue have not been found yet. Though, some relate it to Apple’s move to enhance the text legibility on wallpapers and a darker shade for folders. But, being one of the top favorite feature of the users, OLED Screen, users definitely want Apple to fix this issue as soon as possible.

2. Internet issues on iPhone 8
There have been reports of a potential mobile data issue in some of the iPhone 8 handsets after updating the phone to iOS 12. Some problems with the mobile data connectivity are troubling the users after they have installed the new edition. The main issue is that the phone will disconnect automatically from the mobile data, at least one time every day. The users have to manually connect again to use the internet services. However, some of the users have faced irksome network connectivity problems.

3. Problems with the Bluetooth
Did the Bluetooth icon disappear after updating your iPhone to iOS 12? Well, this is actually not a problem because Apple has purposely removed the icon from the status bar. However, some users are facing issues with the connection to Bluetooth. After switching to iOS 12, the iPad or iPhone users are unable to transfer stuff using Bluetooth connections.

4. Maps are not working appropriately
As ‘Map’ is one of the most useful features of the iPhone, therefore, it is utmost important to make sure that the maps are working absolutely fine. Traffic data is one of the most important parts of the maps. Therefore, maps need to present accurate traffic data to help the users reach their destinations. But, some of the iOS 12 users have seen issues with displaying traffic conditions. Just make sure that the Map settings are apt and you should be able to view the traffic.

5. Alarms aren't functioning
Alarms are used on a daily basis. How difficult it would be to wake up one day and realize that we are 2 hours late simply because the alarm was not working? Some of the iOS 12 users found it difficult to set alarms in their iPhones after they installed iOS 12. Though, people have tried to reboot the phone to make it work. But if it doesn’t work even after that, then you might have to look for a specific solution.

Conclusion
iOS 11 was launched with several bugs, we all know that, but people are thinking that whether iOS 12 has some bugs or not. There can’t be any comparison between the two, but it is believed that Apple wanted to improve the performance more through iOS 12. The focus was less on adding new features, though there are some noteworthy additions. Also, with the new addition, the phones seem to be much more stable. However, there are some bugs in iOS 12 which are still troubling the user. Though, Apple is trying its best to improve and rectify all the issues. And, it has already fixed some bugs with add-on versions of iOS 12 as well.

Machine learning is going to enrich the enterprise mobility methodologies

Nex Mobility — Wed, 25 Jul 2018 13:15:54 +0000

Mobility has transformed the way companies are performing these days. A plenty of discussions which are aimed on the influence of Enterprise mobility on a company have started taking place. This is all because of the rise of phones throughout the organizations. And, this is completing the enterprises relook at their organizational structure entirely, revamping the roles and responsibilities, and nurturing a ‘mobile-centric’ culture shift. Complete adoption of enterprise mobility solutions is not just the application of technology. It covers the understanding the requirements of the employees. And, the organization may have to modify the guidelines and the processes around Enterprise mobility which will eventually revolutionize how a company conducts its core activities.

However, generic enterprise mobility technologies are not enough. Therefore, firms have to keep evolving, and keep integrating newer technologies into the mobility world to make the whole offering all the more fruitful. One of the key technologies which is responsible for positively transforming any field is machine learning and AI. And, it is quickly seeping into the enterprise mobility world to add more value to it as well.

Machine learning is a technology which allows the computers to learn as well as act without being unambiguously programmed. It grows from the understanding of pattern recognition as well as the analysis of the algorithms. This analysis basically allows the learning from data, and thus it makes it possible to steer some of the most useful predictions through the data. It is so omnipresent in the recent time that almost every company is using it, without even realizing the presence of machine learning.

Machine learning is quickly growing as more and more firms are finding ways to lessen the costs, as well as reduce the length of any production life-cycle. For example, a field service worker driving a big rig will use his phone to get a live look at a faulty engine, but with the power of machine learning, it would find the problems via a smart heat sensor which will directly highlight the area which requires mending.

Experts are trying to make the machines more intelligent, so that they are able to assist the employees during the critical tasks. This will also reduce the threat of human replacement as well. A plenty of big brands are already stepping into the machine learning field, and coming up with new tools have the capability to make huge changes in the mobility landscape. With machine learning methodologies, the firms are quickly becoming native inverters as well as the users of the new technologies. And, these technological tools are completely integrating all the processes into their specific workflows and solutions.

Basically, it won’t be wrong if we say that with the integration of machine learning in the enterprise mobility domain we can expect better results. But at the same time, we will also get better revenues at the end. As, the use of mobility along with machine learning is only going to empower the employees and the company to perform better and better. The employees will be able to do their tasks more easily, and with a lot more efficiency. The quality of operations will increase, and a whole new dimension of change will be added in the corporate system. Enterprise mobility is already a great way to make the lives of the workers easier. And, it even helps them to perform better, and to stay connected with the rest of company pretty decently. And, now, that machine learning, AI etc. are making it more powerful, the future of enterprise mobility seems very very bright!

Therefore, machine learning’s integration in the mobility world will not just boost the productivity, but at the end of the day, it is paving way for improved safety. Plus, it is making the complete enterprise mobility setup a lot more advanced and efficient. Therefore, we are sure to find out more ways how machine learning is impacting the mobility world in general as well.