DEV Community: Amit Makhija

How to Integrate the Dual Camera Video Recording Feature in your iOS App?

Amit Makhija — Fri, 20 Sep 2019 10:58:51 +0000

This blog is written with the help of Hitesh Trivedi, who has over 9 years of experience in the iOS app development. He has guided to develop over 100 iPhone apps with unique features and functionalities. He has special expertise in Swift and Objective-C.

Apple announced a new version of OS (iOS13), updates (for instance Siri – Voice Experiences ), technologies and features during the Apple Worldwide Developers Conference (WWDC), 2019. After getting a few queries from our clients about integrating dual recording in their iPhone apps, we decided to write this blog. It is a complete tutorial on how to integrate this feature seamlessly.

What you will learn in this tutorial

How to integrate the dual recording feature in an iOS app

We are an iOS app development company that is always keen on learning and implementing the latest features. Our iOS developers are always experimenting with features to bring out the best in our developed apps. In this tutorial, we have explained how to integrate the multi-camera video recording feature in an iPhone app.

It’s been a little while since the WWDC happened in June this year. Just like every year, Apple made a lot of exciting announcements. A few of these announcements include:

iOS 13
iPad OS
Mac OS Catalina
TVOS 13
Watch OS 6
New MAC Pro and XDR DisplayiOS 13 has generated a lot of buzz recently, and rightly so. It is equipped with a lot of new or updated features and functionalities. Let us see some of the new features of it.

What’s new in iOS 13?

Dark Mode
Camera Capture
Swipe Keyboard
Revamped Reminder App
ARKit 3 (AR Quick Look, People Occlusion, Motion Capture, Reality Composer, track multiple faces)
Siri (Voice Experiences. Shortcuts app)
Sign In with Apple
Multiple UI Instances
Camera Capture(Record a video front/back simultaneously)
Pencil kit
Combine framework
Core Data – Sync Core Data to Cloud kit
Background Tasks

In this iOS tutorial, we will be talking about the Camera Capture functionality where we have used AVMultiCamPiP to capture and record from multiple cameras (front and back).

Multi-camera recording using Camera Capture

Before iOS 13, Apple did not allow to record videos using the front back cameras simultaneously. Now, the users can perform dual camera recording using the back and front camera at the same time. This is done by Camera Capture

PiP in AVMultiCamPiP stands for ‘picture in picture’. This functionality helps to view the video output in full screen and second as the small screen. At any time in between, the user can change the focus.

We created a project called “SODualCamera”, aiming to demonstrate iOS 13 Camera Capture feature on XCode 11. Let’s see the step by step guide to integrate this feature.

Steps to integrate the multi-camera recording feature in an iOS app

1) Create a new project using XCODE 11

2) Select “Single View App” in the iOS section and enter the project name. We have kept it as ‘SODualCamera’.

3) Go to Project Name folder and Open Main.storyboard file. Add Stackview as shown in the figure. In StackView Add two UIView with labels inside UIview.

4) Go to Xcode File Menu and select New and Select File and Select Swift files as shown figure and click the Next button.

5) Now Enter file name as ViewPreview.swift.

6) Now open ViewPreview.swift and add code as below

import AVFoundation
 
class ViewPreview: UIView {
    var videoPreviewLayer: AVCaptureVideoPreviewLayer {
        guard let layer = layer as? AVCaptureVideoPreviewLayer else {
            fatalError("Expected `AVCaptureVideoPreviewLayer` type for layer. Check PreviewView.layerClass implementation.")
        }
        
        layer.videoGravity = .resizeAspect
        return layer
    }
    
    override class var layerClass: AnyClass {
        return AVCaptureVideoPreviewLayer.self  
    }
}

7) Go to Main.storyboard and select StackView. Under StackView, select UIView and add custom class as “ViewPreview” as shown in the figure.

Repeat the same process for the second view.

8) Create outlets for both of View. We are going to use this view as a Preview of camera output.

@IBOutlet weak var backPreview: ViewPreview!
@IBOutlet weak var frontPreview: ViewPreview!

9) As we are aiming to capture video, we need to import the AVFoundation Framework.
We are saving output video to the user’s photo library and we also need to import the Photos framework in ViewController.swift

import AVFoundation
import Photos

10) Declare a variable to perform dual video session.

 var dualVideoSession = AVCaptureMultiCamSession()

11) Create an object of AVCaptureMultiCamSession to execute camera session.

 var audioDeviceInput: AVCaptureDeviceInput?

12) For Audio Device input to record audio while running dual video session, you need to declare variables as follows:

 var backDeviceInput:AVCaptureDeviceInput?
 var backVideoDataOutput = AVCaptureVideoDataOutput()
 var backViewLayer:AVCaptureVideoPreviewLayer?
 var backAudioDataOutput = AVCaptureAudioDataOutput()

 var frontDeviceInput:AVCaptureDeviceInput? 
 var frontVideoDataOutput = AVCaptureVideoDataOutput()
 var frontViewLayer:AVCaptureVideoPreviewLayer?
 var frontAudioDataOutput = AVCaptureAudioDataOutput()

13) In ViewDidAppear add the following code to detect if the app is running on Simulator.

override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)
        
        #if targetEnvironment(simulator)
          let alertController = UIAlertController(title: "SODualCamera", message: "Please run on physical device", preferredStyle: .alert)
          alertController.addAction(UIAlertAction(title: "OK",style: .cancel, handler: nil))
          self.present(alertController, animated: true, completion: nil)
          return
        #endif
    }

14) Create a Setup method to configure the video sessions and also manage user permission. If an app is running on Simulator then return immediately.

#if targetEnvironment(simulator)
            return
        #endif

15) Now check for video recording permission using this code:

switch AVCaptureDevice.authorizationStatus(for: .video) {
            case .authorized:
                // The user has previously granted access to the camera.
                 configureDualVideo()
                break
                
            case .notDetermined:
                
                AVCaptureDevice.requestAccess(for: .video, completionHandler: { granted in
                    if granted{
                        self.configureDualVideo()
                    }
                })
                
                break
                
            default:
                // The user has previously denied access.
            DispatchQueue.main.async {
                let changePrivacySetting = "Device doesn't have permission to use the camera, please change privacy settings"
                let message = NSLocalizedString(changePrivacySetting, comment: "Alert message when the user has denied access to the camera")
                let alertController = UIAlertController(title: "Error", message: message, preferredStyle: .alert)
                
                alertController.addAction(UIAlertAction(title: "OK", style: .cancel, handler: nil))
                
                alertController.addAction(UIAlertAction(title: "Settings", style: .`default`,handler: { _ in
                    if let settingsURL = URL(string: UIApplication.openSettingsURLString) {
                        UIApplication.shared.open(settingsURL,  options: [:], completionHandler: nil)
                    }
                }))
                
                self.present(alertController, animated: true, completion: nil)
            }
        }

16) After getting the user permission to record video, we configure the video session parameters.

First, we need to check whether the device supports MultiCam session

if !AVCaptureMultiCamSession.isMultiCamSupported{
            DispatchQueue.main.async {
               let alertController = UIAlertController(title: "Error", message: "Device is not supporting multicam feature", preferredStyle: .alert)
               alertController.addAction(UIAlertAction(title: "OK",style: .cancel, handler: nil))
               self.present(alertController, animated: true, completion: nil)
            }
            return
        }

17) Now, we set up the front camera first.

func setUpBackCamera() -> Bool{
        //start configuring dual video session
        dualVideoSession.beginConfiguration()
            defer {
                //save configuration setting
                dualVideoSession.commitConfiguration()
            }
 
                
            //search back camera
            guard let backCamera = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else {
                print("no back camera")
                return false
            }
            
            // append back camera input to dual video session
            do {
                backDeviceInput = try AVCaptureDeviceInput(device: backCamera)
                
                guard let backInput = backDeviceInput,dualVideoSession.canAddInput(backInput) else {
                    print("no back camera device input")
                    return false
                }
                dualVideoSession.addInputWithNoConnections(backInput)
            } catch {
                print("no back camera device input: \(error)")
                return false
            }
            
            // seach back video port
            guard let backDeviceInput = backDeviceInput,
                let backVideoPort = backDeviceInput.ports(for: .video, sourceDeviceType: backCamera.deviceType, sourceDevicePosition: backCamera.position).first else {
                print("no back camera input's video port")
                return false
            }
            
            // append back video output
            guard dualVideoSession.canAddOutput(backVideoDataOutput) else {
                print("no back camera output")
                return false
            }
            dualVideoSession.addOutputWithNoConnections(backVideoDataOutput)
            backVideoDataOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)]
            backVideoDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)
            
            // connect back output to dual video connection
            let backOutputConnection = AVCaptureConnection(inputPorts: [backVideoPort], output: backVideoDataOutput)
            guard dualVideoSession.canAddConnection(backOutputConnection) else {
                print("no connection to the back camera video data output")
                return false
            }
            dualVideoSession.addConnection(backOutputConnection)
            backOutputConnection.videoOrientation = .portrait
 
            // connect back input to back layer
            guard let backLayer = backViewLayer else {
                return false
            }
            let backConnection = AVCaptureConnection(inputPort: backVideoPort, videoPreviewLayer: backLayer)
            guard dualVideoSession.canAddConnection(backConnection) else {
                print("no a connection to the back camera video preview layer")
                return false
            }
            dualVideoSession.addConnection(backConnection)
        
        return true
    }

We have now successfully configured the front camera for a video session.

18) We need to follow the same process for the back camera setup.

func setUpBackCamera() -> Bool{
        //start configuring dual video session
        dualVideoSession.beginConfiguration()
            defer {
                //save configuration setting
                dualVideoSession.commitConfiguration()
            }
                
            //search back camera
            guard let backCamera = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back) else {
                print("no back camera")
                return false
            }
            
            // append back camera input to dual video session
            do {
                backDeviceInput = try AVCaptureDeviceInput(device: backCamera)
                
                guard let backInput = backDeviceInput,dualVideoSession.canAddInput(backInput) else {
                    print("no back camera device input")
                    return false
                }
                dualVideoSession.addInputWithNoConnections(backInput)
            } catch {
                print("no back camera device input: \(error)")
                return false
            }
            
            // search back video port
            guard let backDeviceInput = backDeviceInput,
                let backVideoPort = backDeviceInput.ports(for: .video, sourceDeviceType: backCamera.deviceType, sourceDevicePosition: backCamera.position).first else {
                print("no back camera input's video port")
                return false
            }
            
            // append back video output
            guard dualVideoSession.canAddOutput(backVideoDataOutput) else {
                print("no back camera output")
                return false
            }
            dualVideoSession.addOutputWithNoConnections(backVideoDataOutput)
            backVideoDataOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)]
            backVideoDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)
            
            // connect back output to dual video connection
            let backOutputConnection = AVCaptureConnection(inputPorts: [backVideoPort], output: backVideoDataOutput)
            guard dualVideoSession.canAddConnection(backOutputConnection) else {
                print("no connection to the back camera video data output")
                return false
            }
            dualVideoSession.addConnection(backOutputConnection)
            backOutputConnection.videoOrientation = .portrait
 
            // connect back input to back layer
            guard let backLayer = backViewLayer else {
                return false
            }
            let backConnection = AVCaptureConnection(inputPort: backVideoPort, videoPreviewLayer: backLayer)
            guard dualVideoSession.canAddConnection(backConnection) else {
                print("no connection to the back camera video preview layer")
                return false
            }
            dualVideoSession.addConnection(backConnection)
        
        return true
    }
    
        
    func setUpFrontCamera() -> Bool{
            
              //start configuring dual video session
            dualVideoSession.beginConfiguration()
            defer {
              //save configuration setting
                dualVideoSession.commitConfiguration()
            }
            
            //search front camera for dual video session
            guard let frontCamera = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front) else {
                print("no front camera")
                return false
            }
            
            // append front camera input to dual video session
            do {
                frontDeviceInput = try AVCaptureDeviceInput(device: frontCamera)
                
                guard let frontInput = frontDeviceInput, dualVideoSession.canAddInput(frontInput) else {
                    print("no front camera input")
                    return false
                }
                dualVideoSession.addInputWithNoConnections(frontInput)
            } catch {
                print("no front input: \(error)")
                return false
            }
            
            // search front video port for dual video session
            guard let frontDeviceInput = frontDeviceInput,
                let frontVideoPort = frontDeviceInput.ports(for: .video, sourceDeviceType: frontCamera.deviceType, sourceDevicePosition: frontCamera.position).first else {
                print("no front camera device input's video port")
                return false
            }
            
            // append front video output to dual video session
            guard dualVideoSession.canAddOutput(frontVideoDataOutput) else {
                print("no the front camera video output")
                return false
            }
            dualVideoSession.addOutputWithNoConnections(frontVideoDataOutput)
            frontVideoDataOutput.videoSettings = [kCVPixelBufferPixelFormatTypeKey as String: Int(kCVPixelFormatType_32BGRA)]
            frontVideoDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)
            
            // connect front output to dual video session
            let frontOutputConnection = AVCaptureConnection(inputPorts: [frontVideoPort], output: frontVideoDataOutput)
            guard dualVideoSession.canAddConnection(frontOutputConnection) else {
                print("no connection to the front video output")
                return false
            }
            dualVideoSession.addConnection(frontOutputConnection)
            frontOutputConnection.videoOrientation = .portrait
            frontOutputConnection.automaticallyAdjustsVideoMirroring = false
            frontOutputConnection.isVideoMirrored = true
 
            // connect front input to front layer
            guard let frontLayer = frontViewLayer else {
                return false
            }
            let frontLayerConnection = AVCaptureConnection(inputPort: frontVideoPort, videoPreviewLayer: frontLayer)
            guard dualVideoSession.canAddConnection(frontLayerConnection) else {
                print("no connection to front layer")
                return false
            }
            dualVideoSession.addConnection(frontLayerConnection)
            frontLayerConnection.automaticallyAdjustsVideoMirroring = false
            frontLayerConnection.isVideoMirrored = true
            
            return true
    }

19) After Setting up Front and Back Camera, we need to configure for Audio.

First, we need to find the Audio Device Input and then add it to Session. Then find Audio port for front and back camera and add that port to front audio output and video output respectively.

Code looks like this:

func setUpAudio() -> Bool{
         //start configuring dual video session
        dualVideoSession.beginConfiguration()
        defer {
            //save configuration setting
 
            dualVideoSession.commitConfiguration()
        }
        
        // search audio device for dual video session
        guard let audioDevice = AVCaptureDevice.default(for: .audio) else {
            print("no the microphone")
            return false
        }
        
        // append audio to dual video session
        do {
            audioDeviceInput = try AVCaptureDeviceInput(device: audioDevice)
            
            guard let audioInput = audioDeviceInput,
                dualVideoSession.canAddInput(audioInput) else {
                    print("no audio input")
                    return false
            }
            dualVideoSession.addInputWithNoConnections(audioInput)
        } catch {
            print("no audio input: \(error)")
            return false
        }
        
        //search audio port back
        guard let audioInputPort = audioDeviceInput,
            let backAudioPort = audioInputPort.ports(for: .audio, sourceDeviceType: audioDevice.deviceType, sourceDevicePosition: .back).first else {
            print("no front back port")
            return false
        }
        
        // search audio port front
        guard let frontAudioPort = audioInputPort.ports(for: .audio, sourceDeviceType: audioDevice.deviceType, sourceDevicePosition: .front).first else {
            print("no front audio port")
            return false
        }
        
        // append back output to dual video session
        guard dualVideoSession.canAddOutput(backAudioDataOutput) else {
            print("no back audio data output")
            return false
        }
        dualVideoSession.addOutputWithNoConnections(backAudioDataOutput)
        backAudioDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)
        
        // append front output to dual video session
        guard dualVideoSession.canAddOutput(frontAudioDataOutput) else {
            print("no front audio data output")
            return false
        }
        dualVideoSession.addOutputWithNoConnections(frontAudioDataOutput)
        frontAudioDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)
        
        // add back output to dual video session
        let backOutputConnection = AVCaptureConnection(inputPorts: [backAudioPort], output: backAudioDataOutput)
        guard dualVideoSession.canAddConnection(backOutputConnection) else {
            print("no back audio connection")
            return false
        }
        dualVideoSession.addConnection(backOutputConnection)
        
        // add front output to dual video session
        let frontutputConnection = AVCaptureConnection(inputPorts: [frontAudioPort], output: frontAudioDataOutput)
        guard dualVideoSession.canAddConnection(frontutputConnection) else {
            print("no front audio connection")
            return false
        }
        dualVideoSession.addConnection(frontutputConnection)
        
        return true
    }

Now we have successfully configured Front, Back Camera, and Audio.

20) Now when we start a session, it will send output in CMSampleBuffer. We have to collect this sample buffer output and perform the required operation on it.

For this, we need to set the sample delegate method for Camera and Audio. We have already set up delegate for Front, Back Camera and Audio using the following code:

frontVideoDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)
        
  backVideoDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)

   backAudioDataOutput.setSampleBufferDelegate(self, queue: dualVideoSessionOutputQueue)

21) To handle runtime error for a session, we need to add observer as follows:

NotificationCenter.default.addObserver(self, selector: #selector(sessionRuntimeError), name: .AVCaptureSessionRuntimeError,object: dualVideoSession)
        
        NotificationCenter.default.addObserver(self, selector: #selector(sessionWasInterrupted), name: .AVCaptureSessionWasInterrupted, object: dualVideoSession)
        
        NotificationCenter.default.addObserver(self, selector: #selector(sessionInterruptionEnded), name: .AVCaptureSessionInterruptionEnded, object: dualVideoSession)

22) So now we are ready to launch the session. We are going to use a different queue for the session as when we run the session on the main thread, it causes performance issues, lagging, memory leaking.

Start the session using this code:

dualVideoSessionQueue.async {
          self.dualVideoSession.startRunning()
      }

23) Now, we will Add Gesture recognizer for starting and stopping the recording in the setup method as follows:

func addGestures(){
   
        let tapSingle = UITapGestureRecognizer(target: self, action: #selector(self.handleSingleTap(_:)))
        tapSingle.numberOfTapsRequired = 1
        self.view.addGestureRecognizer(tapSingle)
        
 
        let tapDouble = UITapGestureRecognizer(target: self, action: #selector(self.handleDoubleTap(_:)))
        tapDouble.numberOfTapsRequired = 2
        self.view.addGestureRecognizer(tapDouble)
        
        tapSingle.require(toFail: tapDouble) 
//differentiate single tap and double tap recognition if the user do both gestures simultaneously. 
 
}

24) In Apple Capture demo, they have used AVCapture Delegate method to fetch front and back video.

Here, we are not going to use this Sample Output Delegate method. We are going to use ReplayKit to record it. We will do screen recording to generate output video here.

25) Import Replaykit framework as follows:

 import ReplayKit

26) Create one shared object of RPScreenRecorder, to start and stop screen recording.

 let screenRecorder = RPScreenRecorder.shared()

27) We will track recording status using isRecording variable.

 var isRecording = false

28) When the user taps on the screen once, it starts screen recording and when user double taps on screen it stops screen recording and saves output video to the user’s photo library.

We have used one custom class for appending video and audio output to one video output using AssetWriter.swift

import UIKit
import Foundation
import AVFoundation
import ReplayKit
import Photos
 
extension UIApplication {
 
    class func getTopViewController(base: UIViewController? = UIApplication.shared.keyWindow?.rootViewController) -> UIViewController? {
 
        if let nav = base as? UINavigationController {
            return getTopViewController(base: nav.visibleViewController)
 
        } else if let tab = base as? UITabBarController, let selected = tab.selectedViewController {
            return getTopViewController(base: selected)
 
        } else if let presented = base?.presentedViewController {
            return getTopViewController(base: presented)
        }
        return base
    }
}
 
class AssetWriter {
    private var assetWriter: AVAssetWriter?
    private var videoInput: AVAssetWriterInput?
    private var audioInput: AVAssetWriterInput?
    private let fileName: String
    
    let writeQueue = DispatchQueue(label: "writeQueue")
    
    init(fileName: String) {
        self.fileName = fileName
    }
    
    private var videoDirectoryPath: String {
        let dir = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]
        return dir + "/Videos"
    }
    
    private var filePath: String {
        return videoDirectoryPath + "/\(fileName)"
    }
    
    private func setupWriter(buffer: CMSampleBuffer) {
        if FileManager.default.fileExists(atPath: videoDirectoryPath) {
            do {
                try FileManager.default.removeItem(atPath: videoDirectoryPath)
            } catch {
                print("fail to removeItem")
            }
        }
        do {
            try FileManager.default.createDirectory(atPath: videoDirectoryPath, withIntermediateDirectories: true, attributes: nil)
        } catch {
            print("fail to createDirectory")
        }
        
        self.assetWriter = try? AVAssetWriter(outputURL: URL(fileURLWithPath: filePath), fileType: AVFileType.mov)
        
        let writerOutputSettings = [
            AVVideoCodecKey: AVVideoCodecType.h264,
            AVVideoWidthKey: UIScreen.main.bounds.width,
            AVVideoHeightKey: UIScreen.main.bounds.height,
            ] as [String : Any]
        
        self.videoInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: writerOutputSettings)
        self.videoInput?.expectsMediaDataInRealTime = true
        
        guard let format = CMSampleBufferGetFormatDescription(buffer),
            let stream = CMAudioFormatDescriptionGetStreamBasicDescription(format) else {
                print("fail to setup audioInput")
                return
        }
        
        let audioOutputSettings = [
            AVFormatIDKey : kAudioFormatMPEG4AAC,
            AVNumberOfChannelsKey : stream.pointee.mChannelsPerFrame,
            AVSampleRateKey : stream.pointee.mSampleRate,
            AVEncoderBitRateKey : 64000
            ] as [String : Any]
        
        self.audioInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioOutputSettings)
        self.audioInput?.expectsMediaDataInRealTime = true
        
        if let videoInput = self.videoInput, (self.assetWriter?.canAdd(videoInput))! {
            self.assetWriter?.add(videoInput)
        }
        
        if  let audioInput = self.audioInput, (self.assetWriter?.canAdd(audioInput))! {
            self.assetWriter?.add(audioInput)
        }
    }
    
    public func write(buffer: CMSampleBuffer, bufferType: RPSampleBufferType) {
        writeQueue.sync {
            if assetWriter == nil {
                if bufferType == .audioApp {
                    setupWriter(buffer: buffer)
                }
            }
            
            if assetWriter == nil {
                return
            }
            
            if self.assetWriter?.status == .unknown {
                print("Start writing")
                let startTime = CMSampleBufferGetPresentationTimeStamp(buffer)
                self.assetWriter?.startWriting()
                self.assetWriter?.startSession(atSourceTime: startTime)
            }
            if self.assetWriter?.status == .failed {
                print("assetWriter status: failed error: \(String(describing: self.assetWriter?.error))")
                return
            }
            
            if CMSampleBufferDataIsReady(buffer) == true {
                if bufferType == .video {
                    if let videoInput = self.videoInput, videoInput.isReadyForMoreMediaData {
                        videoInput.append(buffer)
                    }
                } else if bufferType == .audioApp {
                    if let audioInput = self.audioInput, audioInput.isReadyForMoreMediaData {
                        audioInput.append(buffer)
                    }
                }
            }
        }
    }
    
    public func finishWriting() {
        writeQueue.sync {
            self.assetWriter?.finishWriting(completionHandler: {
                print("finishWriting")
                PHPhotoLibrary.shared().performChanges({
                    PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: URL(fileURLWithPath: self.filePath))
                }) { saved, error in
                    if saved {
                        let alertController = UIAlertController(title: "Your video was successfully saved", message: nil, preferredStyle: .alert)
                        let defaultAction = UIAlertAction(title: "OK", style: .default, handler: nil)
                        alertController.addAction(defaultAction)
                        if let topVC = UIApplication.getTopViewController() {
                            topVC.present(alertController, animated: true, completion: nil)
                        }
                    }
                }
            })
        }
    }
}

29) Now we can go to the final stage of our demo.

When the user taps on screen we need to start the screen recording.

Create Start Capture function as following:

func startCapture() {
       screenRecorder.startCapture(handler: { (buffer, bufferType, err) in
            self.isRecording = true
            self.assetWriter!.write(buffer: buffer, bufferType: bufferType)
        }, completionHandler: {
            if let error = $0 {
                print(error)
            }
        })
    }

Method of Screen recorder will return as buffer and buffer type, which we need to pass to the assetwriter. AssetWriter will write this buffer into the URL, which we need to pass to assetwriter initially.

30) We have created an object of AssetWriter assetWrite and initialized as follows:

let outputFileName = NSUUID().uuidString + ".mp4"
   
assetWriter = AssetWriter(fileName: outputFileName)

31) Now when the user double-taps on the screen, screen recording stops. So we will use stopCapture of screen recorder method as following:

func stopCapture() {
        screenRecorder.stopCapture {
            self.isRecording = false
            if let err = $0 {
                print(err)
            }
            self.assetWriter?.finishWriting()
        }
    }

32) As we are not writing any buffer into the assetWriter, we need to tell assetWriter to finish writing and generate output video URL.

Now AssetWriter will finish writing and it will ask the user for permission to store video in the user photo library and if the user approves permission it will be saved there.

AssetWriter has finishWritng function for doing this.

public func finishWriting() {
        writeQueue.sync {
            self.assetWriter?.finishWriting(completionHandler: {
                print("finishWriting")
                PHPhotoLibrary.shared().performChanges({
                    PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: URL(fileURLWithPath: self.filePath))
                }) { saved, error in
                    if saved {
                        let alertController = UIAlertController(title: "Your video was successfully saved", message: nil, preferredStyle: .alert)
                        let defaultAction = UIAlertAction(title: "OK", style: .default, handler: nil)
                        alertController.addAction(defaultAction)
                        if let topVC = UIApplication.getTopViewController() {
                            topVC.present(alertController, animated: true, completion: nil)
                        }
                    }
                }
            })
        }
    }

Note: Some developers might get confused that RPscreenRecorder also provides startRecording and stopRecording but we have used startCapture and stopCapture.

Why is it so?

Here’s an image to answer it

startRecording will start recording the app display. When we use startRecording method, to end recording we need to call stopRecroding method of RPscreenRecorder.

When we are running an AVSession and want to record screen, at that time we need to use startCapture method.

startCapture will start screen and audio capture. The recording initiated with startCapture must end with stopCapture.

Summing up

We hope that you found this iPhone tutorial useful and your concepts about iPhone dual camera recording are clear. You can find the source code of this record front and back camera same time iPhone illustration on Github.

iOS Tutorial: How to customize dark mode in iOS app with iOS 13

Amit Makhija — Tue, 30 Jul 2019 11:16:22 +0000

Apple Worldwide Developers Conference (WWDC), 2019 brought a lot of new features, technologies and a new version of iOS (iOS 13) as well. It was organized from 3rd to 7th June 2019. During this developers’ conference, it was announced that Dark Mode will be available on iOS 13 and later versions.

What you will learn in this blog:

What iOS Dark Mode is and what it does
The benefits of using Dark Mode
Important considerations while implementing Dark Mode in an app
Customizing the color scheme for your iOS application for different themes – light and dark mode.

What is iOS Dark Mode? What does it do?

Dark Mode is a dark system-wide appearance that uses a darker color palette for all screens, views, menus, and controls. Basically, like Android Dark Mode, it changes a bright theme to a darker one. It also maintains vibrancy and contrast to make foreground content stand out against a darker background.

Benefits of Dark Mode

It is most helpful when the surrounding is dark. This puts less strain on eyes as compared to light mode.
You can choose Dark Mode as your default interface style.
You can also use Settings to make your device switch to Dark Mode automatically when ambient light is low.
Dark Mode supports all accessibility features.
Using Dark Mode on OLED iPhone models like iPhone X, XS, XS Max may also improve battery life. This is because black pixels in an OLED panel actually switch off and conserve battery power.
Bonus: Dark Mode looks really cool

As announced by Apple during the conference, Apple’s preinstalled apps like Notes, News, TV, Music, Reminders, Mail, and others will already have it.

However, developers will be able to use Apple’s new tools to add dark mode to their apps. Being an iPhone app development company, we had a lot of our clients wanting to integrate this feature to their iOS apps.

Our proficient iPhone app developers decided to help other developers by explaining the feature with a simple example. Before we see how to add this feature, let us know which are the important considerations that one should keep in mind during feature integration.

Considerations for iPhone Dark Mode while implementing in an app

Put the focus on content: Make sure that content stands out and surrounding UI recedes into the background. The purpose of the Dark Mode is to put the focus on the content areas of the interface.

Test for both-light and dark appearances: Check how the interface looks in both appearances. You may need to adjust designs to accommodate each one as designs working well in one appearance might not work in another.

Test with changes in contrast and transparency: You must test the content when Increase Contrast and Reduce Transparency is turned on. Make sure that content is distinct in Dark Mode when you adjust contrast and transparency settings.

Now that you are clear about considerations, let’s customize the color scheme for your iOS application for different themes.

How to customize the color scheme for your iOS application for different themes – light and dark mode.

Step 1: Create a new project in Xcode

Step 2: Add a new color set in assets.xcassets folder as shown in the images

Step 3: Go to Storyboard and drag & drop controls that you would like to add to your project.

Step 4: Go to the view controller and create outlet(s) for UI Controls.

Step 5: Create employee struct and create an array of employee struct

Step 6: Append the data in arrEmployee to bind with UITableView

Step 7: Assign data source for the table view

Step 8: Create a UITableViewCell class for employee

Step 9: Implement the UITableView data source method

Step 10: Create switch event to set theme ( light or dark ) for UILable

Step 11: Set default behavior for controls in viewdidload

Step 12: compare assets folder with below images

That’s all folks!

Outcome

Concluding notes

We hope that you found this iPhone tutorial useful and your concepts about iPhone Dark Mode are clear. You can find the source code of this iOS 13 Dark Mode illustration on github.

Here, in this iPhone app tutorial, we learned about customizing the iOS app theme for both ios dark theme and light.

If you have any suggestions or queries in this tutorial or any questions regarding iPhone app development, we are all ears.

How to use BackgroundTasks framework to keep your iOS app content up to date?

Amit Makhija — Fri, 19 Jul 2019 10:06:53 +0000

In this technical blog, we are going to understand how to schedule iOS background tasks. This iOS tutorial is for those who want to implement the background task scheduler in their latest iOS app development project.

What you will learn?

In this iOS tutorial, you’re going to see an example of using the new BackgroundTasks framework for fetching images in the background, while the phone is in idle mode.

Last month at WWDC Developer Conference 2019, Apple released the latest iOS 13 with a huge list of new features and functionalities. It was the second major and one of the most important revisions of an Operating System which is being used on more than 1 Billion iOS devices worldwide.

The most significant thing that Apple has brought is the Optimized Trend, which was launched with iOS 12, now making iOS 13 faster & more efficient. The app updating time has been improved, while the app launching time has become 2x quicker. Also, the app download size has been reduced to half (50%).

Let’s talk about some new features and functionalities of iOS 13 and then move on to understand one of the most important functionalities- BackgroundTasks framework.

Here’s a list of some of the important features brought in iOS 13:

Dark Mode
Revamped Photos app
Sign In with Apple (which we already covered in detail with illustration)
HomeKit Secure Video
Name and image in Messages
Swiping keyboard
Multi-user HomePod
All-new Reminders app
Memoji and stickers
Smarter, smoother Siri voice assistance

In terms of technical functionalities, Apple has introduced:

Advances in contextual action menus in iOS, macOS and iPadOS
UIWindowScene API and multitasking in iPadOS
AI/ML functionalities like Image and Speech saliency, Word embeddings, Sound analysis, Text catalog and recognition, Image similarity & classification, On-device speech, Face capture quality, Sentiment classification
Conversational shortcuts in Siri for apps
New BackgroundTasks Framework

In this iOS app tutorial, we will talk about the “BackgroundTasks Framework”.

BackgroundTasks Framework

This new framework is used for tasks like cleaning a database, updating a machine learning model, or updating the displayed data for an app, and other deferrable tasks that are better done in the background. It makes efficient use of processing time and power, and run tasks like these when the device is in idle condition.

BackgroundTasks Framework has two main task requests under BGTaskScheduler:

BGAppRefreshTaskRequest: This is a request to launch an app in the background to execute a short refresh task.
BGProcessingTaskRequest: This is a request to launch an app in the background and execute a process that takes a longer time to complete.

BackgroundTasks can be used to perform various activities like database cleaning, uploading pictures to a server, syncing pictures in other devices, and many more.

Our expert iOS developers at Space-O Technologies received a lot of queries from our clients and other developers when BackgroundTasks Framework was introduced. Our developers were happy to help them and decided to come up with a small iOS demo about iOS background image fetching, to explain a few things.

In this iOS tutorial, we are going to take the iOS background task example of fetching the latest count of added images in the image gallery.

Fetching Image Count While Processing Task In Background

Implementing BackgroundTasks Framework in your project

1) Create a new project using XCODE 11.

2) Select “Single View App” in the iOS section and enter the project name. (We have kept the project name as “SOBackgroundTask”).

3) Go to SoBackgroundTask Target and click on “Signing & Capabilities”, then click on “+ Capability”,

4) Double-tap on “Background Modes”

5) Select “Background Fetch” and “Background Processing” from all background tasks.

6) Add “BGTaskSchedulerPermittedIdentifiers” key in info.plist and add a task identifier array.

Note: The system only runs the tasks registered with identifiers on a whitelist of task identifiers.

7) import BackgroundTasks in AppDelegate.swift.

8) Create registerBackgroundTaks() method with identifier (use the same identifier we used in info.plist) and call it from Application:didFinishLaunchingWithOptions

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {

registerBackgroundTaks()
return true
}

//MARK: Register BackGround Tasks
private func registerBackgroundTaks() {

BGTaskScheduler.shared.register(forTaskWithIdentifier: "com.SO.imagefetcher", using: nil) { task in
//This task is cast with processing request (BGProcessingTask)
self.scheduleLocalNotification()
self.handleImageFetcherTask(task: task as! BGProcessingTask)
}

BGTaskScheduler.shared.register(forTaskWithIdentifier: "com.SO.apprefresh", using: nil) { task in
//This task is cast with processing request (BGAppRefreshTask)
self.scheduleLocalNotification()
self.handleAppRefreshTask(task: task as! BGAppRefreshTask)
}
}

9) Create scheduleImagefetcher() and scheduleAppRefresh() method for fetching images from the gallery and refresh app once image fetch is completed. These methods are called from applicationDidEnterBackground.

func applicationDidEnterBackground(_ application: UIApplication) {
scheduleAppRefresh()
scheduleImagefetcher()
}

func scheduleImagefetcher() {
let request = BGProcessingTaskRequest(identifier: "com.SO.imagefetcher")
request.requiresNetworkConnectivity = false // Need to true if your task need to network process. Defaults to false.
request.requiresExternalPower = false
//If we keep requiredExternalPower = true then it required device is connected to external power.

request.earliestBeginDate = Date(timeIntervalSinceNow: 1 * 60) // fetch Image Count after 1 minute.
//Note :: EarliestBeginDate should not be set to too far into the future.
do {
try BGTaskScheduler.shared.submit(request)
} catch {
print("Could not schedule image fetch: (error)")
}
}

func scheduleAppRefresh() {
let request = BGAppRefreshTaskRequest(identifier: "com.SO.apprefresh")
request.earliestBeginDate = Date(timeIntervalSinceNow: 2 * 60) // App Refresh after 2 minute.
//Note :: EarliestBeginDate should not be set to too far into the future.
do {
try BGTaskScheduler.shared.submit(request)
} catch {
print("Could not schedule app refresh: (error)")
}
}

Note: You need to cancel pending background tasks if any, otherwise it will display error code=2.

To cancel pending background task, we call the below method before scheduling a new task.

func cancelAllPendingBGTask() {
BGTaskScheduler.shared.cancelAllTaskRequests()
}

Note: iOS background task time limit is 30 seconds, the policy is still the same.

Let’s Sum Up!

We hope this iOS tutorial has helped you to understand how BackgroundTasks Framework works. You can get the source code by referring to the github demo.

Here, we took only one example of fetching images in the background, there are various tasks for which we can use this framework.

Let us know if you have any suggestions or queries in this tutorial or any questions regarding iPhone app development. We are one of the leading iPhone app development companies and already developed over 2500 iOS apps successfully.

So, if you have an iPhone app idea or want to create a mobile application on the iOS platform with advanced features and functionalities like this one, discuss it with us through our contact form. We are all ears!

Android Tutorial: How to reduce Android app size?

Amit Makhija — Thu, 11 Jul 2019 05:24:24 +0000

Do you know about 70% of people in emerging markets consider the size of an app before downloading it because they are concerned about data cost and phone storage space.? Are you facing trouble while uploading your apps because of their large size? Here’s a tutorial explaining how you can reduce android app size at the developer’s level as well as how Android App Bundle makes this task easy.

What you will learn?

How to reduce android app size at the developer’s level by,

Removing unused resources, code, and classes and reusing resources wherever possible.
Reducing the size of images
Reducing the size of native and Java codebase
A brief explanation of the role of Android app bundle in reducing app size

If the application is large, people would rather not download it all. Let’s imagine that this message pops up when a user installs your app.

The user thinks they’re probably better off without the app. Who would go through all the trouble of finding and deleting unused files and apps for just one app? Even if they have enough storage but the internet speed is slow, they might get bored of the time it is taking to download and stop the process in between.

This is why it is extremely important to develop an app with high performance, quality design and seamless user experience that also doesn’t take up too much space.

The question is – how?

Android developers at Space-O Technologies receive a lot of queries about how they build effective small-sized apps, and this inspired us to write a practical Android tutorial on how to reduce Android app size. We will first get to the methods that a developer should take while developing an app and then go on to understand about Android App Bundle.

Before reducing app size, let’s analyze app size using APK analyzer and understand the APK structure.

Directories

META-INF/ - Contains the CERT.SF and CERT.RSA signature files, as well as the MANIFEST.MF manifest file.
assets/ - All the app’s assets are contained by this. The app can retrieve them using an AssetManager object.
res/ - This contains resources that aren’t compiled into resources.arsc.
lib/ - This contains the compiled code that is specific to the software layer of a processor.

Files

resources.arsc - It contains compiled resources. All the language strings and styles, layout files and images are included in this. Basically, it contains the XML content from all configurations of the res/values/ folder.

classes.dex - It contains the classes compiled in the DEX file format understood by the Dalvik/ART virtual machine. You may have multiple .dex files if the number of methods is beyond 64K

AndroidManifest.xml: It has the core Android manifest file that lists the name, version, access rights, and referenced library files of the app. The file uses Android’s binary XML format.

How to reduce Android app size or APK size

1.Remove unused resources, code, and classes

The lint tool detects resources in the res/ folder that your code doesn’t reference. The lint tool only detects potentially unused resources and prints a message like this,

res/layout/preferences.xml: Warning: The resource R.layout.preferences appears
to be unused [UnusedResources]

Enable resource shrinking and shrink code using proguard. By setting minifyEnabled to true, proguard removes all the unused methods and slims down the classes.dex file.

You can also reuse resources. Instead of using different resources, utilize one resource in different manners. Here is an example from the Android Developers site,

<?xml version="1.0" encoding="utf-8"?>
android:drawable="@drawable/ic_thumb_up"
android:pivotX="50%"
android:pivotY="50%"
android:fromDegrees="180" />

If your application requires only one language, for example, English, then you should use resConfig. Enabling resConfig will only add English resources in your apk file. Here is how you can do it,

2.Reduce the size of images

This can be done in many different ways, but we will discuss only a few main ones

a) You could use drawable shapes like this

b) Use .webp image format: Convert the .jpg and .png images to .webp format. You can see that there is no reduction in image quality but the size is considerably reduced.

c) Compress PNG and JPG files

To compress PNG files, you may use tools like pngcrush, pngquant, or zopflipng.

To compress JPG files, tools like packJPG and guetzli are very effective.

d) Use vector graphics for simple images as they utilize space

e) Use 9-patch images to save a lot of space. The Draw 9-patch tool is used to create bitmap images that automatically resize to accommodate the contents of the view and the size of the screen.

3.Reduce the size of native and Java codebase

This can be done in several ways,

Remove unnecessary code that is generated automatically.
Avoid enumerations: Use the @IntDef annotation and ProGuard to strip enumerations out and convert them to integers.
Reduce the size of native binaries: Two ways to do this

– Remove debug symbols: Use the arm-eabi-strip tool, provided in the Android NDK, to remove unnecessary debug symbols. Always publish your app in the release version.

– Avoid extracting native libraries: By setting android:extractNativeLibs=”false” prevents PackageManager from copying .so files from the APK to the filesystem during installation. This makes updates of your app smaller.

This is what a developer can do on his level, but there is a lot more in store for Android app developers. We’re talking about the Android app bundle format. What’s that?

With an increase in features, Android app sizes kept increasing which led to a decrease in the number of installs. This is why Android studio came up with Android App Bundles.

What is the Android App Bundle (.aab) format? Why use Android App Bundle?

What is the Android App Bundle format?

Initially, developers could publish their apps in the .apk format in two ways,

Either as a single file with all the code and resources for the different device configurations that your app supports.
Or as multiples files individually handling different device configurations.

Publishing an app with .aab format is the third option that is provided by the Android studio. Android app bundle is the new publishing format for Android apps. It is the best format if you want to reduce Android app size.

Why use Android App Bundle?

Android App Bundle serves only the code and resources a user needs to run your app on their specific device. Using this format you could have 35% size savings as compared to universal APK. You don’t need to use incomplete solutions like multi-APK thus saving you a lot of time and effort.

The app bundle uses the Dynamic Delivery serving model to reduce Android app size. App size is reduced in two main ways,

1.Using Configuration APK

Upload your app with the .aab format. Google Play will use this to generate 2 APKs: a single Base APK and multiple Configuration APKs. Base APK contains all the code and resources required to deliver your app’s base functionality. A user will first receive this APK and ever subsequent APK will depend on this. This is generated from your project’s app or base module.

After this, Google Play will scan the device configuration to generate Configuration APKs. Every time, the app is downloaded on a device, the configuration is scanned and Dynamic Delivery model comes into play. Only a customized configuration APK is downloaded on the device with respect to its individual configuration.

Thus only limited space is required for the app.

2.Using on-demand features

If you wish to create a modular app, like a gaming app, then this one is for you. There are features in your application not required by many of your users, they may only download required features. You can distribute your features into dynamic feature modules which decreases the size of your APK, making it a dynamic Android app.

When a user requires a feature of one of these modules, he will install apks and Dynamic Delivery will serve a dynamic feature APK compiling the resources and code to run that specific feature.

Let’s Sum Up!

In this Android tutorial, we learned how to,

Remove unused resources, code, and classes. If possible, reuse resources wherever possible.
Reduce the size of images using drawable shapes, vector images, 9-patch images or convert to webp format or compress png and jpg images
Reduce the size of native and Java codebase
Use Android app bundle

We hope that this tutorial helps you to reduce your app size and you consider to develop small-sized feature-rich apps.

If you know any other simpler or faster way to reduce Android app size, please share it in the comment section.