DEV Community

Cover image for CoreML Tutorial Swift: From Basics to Apple Foundation Models
Iniyarajan
Iniyarajan

Posted on

CoreML Tutorial Swift: From Basics to Apple Foundation Models

#coreml #swift #iosai #applefoundationmodels

Over 87% of iOS developers are now implementing some form of on-device AI in their apps — but many still struggle with CoreML's learning curve and the transition to Apple's new Foundation Models framework. If you're one of those developers searching for a practical CoreML tutorial Swift guide that bridges traditional machine learning with Apple's latest AI capabilities, you're in the right place.

CoreML Swift development
Photo by Jovan Vasiljević on Pexels

Table of Contents

Understanding CoreML's Evolution in 2026

CoreML has undergone significant changes since Apple introduced Foundation Models at WWDC 2026. You now have access to both traditional CoreML models for vision and audio tasks, plus Apple's on-device language models for text generation. This dual approach gives you unprecedented flexibility in building AI-powered iOS apps.

Related: On Device ML iOS: Apple's Foundation Models vs CoreML in 2026

The key difference lies in their use cases. Traditional CoreML excels at classification, object detection, and structured prediction tasks. Apple's Foundation Models framework handles natural language processing, text generation, and conversational AI — all running locally on devices with A17 Pro chips or newer.

System Architecture

Setting Up Your First CoreML Project

Your CoreML tutorial Swift journey begins with proper project setup. Create a new iOS project in Xcode and ensure you're targeting iOS 15.0 or later for broad CoreML compatibility, or iOS 26.0+ if you plan to use Foundation Models.

Also read: AI Integration Mobile Apps Swift: iOS 26 Foundation Models

First, import the necessary frameworks:

import UIKit
import CoreML
import Vision
import VisionKit
// For Foundation Models (iOS 26+)
import FoundationModels
Enter fullscreen mode Exit fullscreen mode

Next, add your model file to your project bundle. You can download pre-trained models from Apple's Machine Learning gallery or create custom ones using Create ML. The most common beginner-friendly model is MobileNetV2 for image classification.

Here's the basic setup for a CoreML-enabled view controller:

class MLViewController: UIViewController {

    private var model: VNCoreMLModel?

    override func viewDidLoad() {
        super.viewDidLoad()
        setupCoreMLModel()
    }

    private func setupCoreMLModel() {
        guard let modelURL = Bundle.main.url(forResource: "MobileNetV2", withExtension: "mlmodelc") else {
            print("Failed to find model file")
            return
        }

        do {
            let model = try VNCoreMLModel(for: MLModel(contentsOf: modelURL))
            self.model = model
        } catch {
            print("Failed to load model: \(error)")
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Building a Basic Image Classification App

Now you'll create a practical image classification app that demonstrates CoreML's capabilities. This example processes camera input and provides real-time predictions.

Process Flowchart

Implement the image processing pipeline:

private func classifyImage(_ image: UIImage) {
    guard let model = self.model,
          let ciImage = CIImage(image: image) else { return }

    let request = VNCoreMLRequest(model: model) { [weak self] request, error in
        guard let results = request.results as? [VNClassificationObservation] else { return }

        DispatchQueue.main.async {
            self?.displayResults(results)
        }
    }

    let handler = VNImageRequestHandler(ciImage: ciImage, options: [:])
    do {
        try handler.perform([request])
    } catch {
        print("Failed to perform classification: \(error)")
    }
}

private func displayResults(_ results: [VNClassificationObservation]) {
    guard let topResult = results.first else { return }

    let confidence = Int(topResult.confidence * 100)
    let text = "\(topResult.identifier) (\(confidence)%)"

    // Update your UI with the prediction
    resultLabel.text = text
}
Enter fullscreen mode Exit fullscreen mode

Integrating Apple Foundation Models

Apple's Foundation Models framework represents the future of on-device AI for iOS apps. You can now generate text, analyze sentiment, and perform complex language tasks without sending data to external servers.

Here's how you integrate the SystemLanguageModel for text generation:

import FoundationModels

class AITextGenerator {
    private let model = SystemLanguageModel.default

    func generateResponse(to prompt: String) async throws -> String {
        let request = LanguageModelRequest(
            prompt: prompt,
            maxTokens: 150,
            temperature: 0.7
        )

        let response = try await model.generate(request)
        return response.text
    }

    // Using @Generable for structured output
    @Generable
    struct ProductReview {
        let rating: Int
        let sentiment: String
        let summary: String
    }

    func analyzeReview(_ reviewText: String) async throws -> ProductReview {
        let prompt = "Analyze this product review: \(reviewText)"
        return try await model.generate(ProductReview.self, prompt: prompt)
    }
}
Enter fullscreen mode Exit fullscreen mode

The @Generable macro automatically creates the necessary schema for guided generation, ensuring your model output matches your Swift types exactly. This eliminates the need for manual JSON parsing and type conversion.

Performance Optimization Strategies

Optimizing your CoreML implementation is crucial for maintaining smooth user experiences. You need to consider model size, inference speed, and memory usage.

Model Optimization Techniques:

  1. Quantization: Reduce model size by using 16-bit or 8-bit weights instead of 32-bit floats
  2. Pruning: Remove unnecessary neural network connections
  3. Batch Processing: Process multiple inputs simultaneously when possible

Memory Management Best Practices:

class OptimizedMLManager {
    private var modelCache: [String: VNCoreMLModel] = [:]
    private let maxCacheSize = 3

    func getModel(named name: String) -> VNCoreMLModel? {
        // Check cache first
        if let cachedModel = modelCache[name] {
            return cachedModel
        }

        // Load and cache model
        guard let newModel = loadModel(named: name) else { return nil }

        // Manage cache size
        if modelCache.count >= maxCacheSize {
            let oldestKey = modelCache.keys.first!
            modelCache.removeValue(forKey: oldestKey)
        }

        modelCache[name] = newModel
        return newModel
    }
}
Enter fullscreen mode Exit fullscreen mode

For Foundation Models, you can use LoRA (Low-Rank Adaptation) to fine-tune models efficiently:

// Fine-tune with LoRA adapters
let adapter = try await LoRAAdapter.load(from: Bundle.main.url(forResource: "custom-adapter", withExtension: "lora")!)
let customizedModel = model.applying(adapter)
Enter fullscreen mode Exit fullscreen mode

Performance Monitoring:

Implement timing measurements to track inference performance:

func measureInferenceTime<T>(_ operation: () throws -> T) rethrows -> (result: T, time: TimeInterval) {
    let startTime = CFAbsoluteTimeGetCurrent()
    let result = try operation()
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    return (result, timeElapsed)
}
Enter fullscreen mode Exit fullscreen mode

Frequently Asked Questions

Q: How do I convert a TensorFlow model to CoreML format?

Use the coremltools Python package: pip install coremltools, then convert with coremltools.convert(model, inputs=[...]). The converted .mlmodel file can be directly imported into your Xcode project.

Q: What's the difference between CoreML and Apple's Foundation Models?

CoreML handles traditional ML tasks like image classification and audio processing using custom-trained models. Foundation Models provide pre-trained language capabilities for text generation, analysis, and conversation — think ChatGPT but running entirely on-device.

Q: Can I use CoreML models offline without internet connectivity?

Yes, that's CoreML's primary advantage. All inference happens on-device using the Neural Engine or GPU. Your app works completely offline once the model is bundled with your app.

Q: How do I handle model updates in production iOS apps?

Implement a model versioning system using CloudKit or your backend. Download updated models in the background and swap them during app launches. Always include a fallback mechanism in case model loading fails.

Need a server? Get $200 free credits on DigitalOcean to deploy your AI apps.

Resources I Recommend

If you're serious about mastering iOS AI development, this collection of Swift programming books provides the foundation you need for advanced CoreML implementations. For deeper AI understanding, these AI and LLM engineering books cover the theoretical concepts behind Apple's Foundation Models framework.

You Might Also Like

Building AI-powered iOS apps in 2026 means mastering both traditional CoreML for specialized tasks and Apple's Foundation Models for language processing. Start with simple image classification to understand the CoreML workflow, then gradually incorporate Foundation Models for more sophisticated AI features. The key is understanding when to use each framework and how to optimize performance for the best user experience.

📘 Go Deeper: AI-Powered iOS Apps: CoreML to Claude

200+ pages covering CoreML, Vision, NLP, Create ML, cloud AI integration, and a complete capstone app — with 50+ production-ready code examples.

Get the ebook →

Also check out: *Building AI Agents***

Enjoyed this article?

I write daily about iOS development, AI, and modern tech — practical tips you can use right away.

  • Follow me on Dev.to for daily articles
  • Follow me on Hashnode for in-depth tutorials
  • Follow me on Medium for more stories
  • Connect on Twitter/X for quick tips

If this helped you, drop a like and share it with a fellow developer!

Top comments (0)