🚀 Harbeth: High-Performance Swift Image Processing Library
The Journey Behind Harbeth
As an iOS/macOS developer, I've always been fascinated by the power of GPU-accelerated image processing. After working with various libraries and facing performance bottlenecks in real-time applications, I decided to create Harbeth - a library that combines the best of Metal performance with an intuitive Swift API.
The Challenges I Faced
- Performance Limitations: Existing solutions either lacked performance or had complex APIs
- Cross-Platform Consistency: Ensuring the same code worked seamlessly across iOS, macOS, tvOS, and watchOS
- Memory Management: Optimizing texture usage to avoid memory spikes during real-time processing
- API Design: Creating an intuitive interface that hides the complexity of Metal while providing full control
What Makes Harbeth Special
After months of development and optimization, Harbeth has become a library that I'm truly proud of. Let me share what makes it stand out:
🎨 Technical Deep Dive
Metal-Powered Architecture
Harbeth is built on a layered architecture that maximizes Metal's capabilities:
// Core rendering pipeline
class C7RenderPipeline {
private let device: MTLDevice
private let commandQueue: MTLCommandQueue
private let library: MTLLibrary
func render(with filter: C7FilterProtocol, inputTexture: MTLTexture) -> MTLTexture {
// 1. Create render pass descriptor
// 2. Set up command buffer
// 3. Encode render commands
// 4. Execute and return output texture
}
}
Smart Texture Pooling
One of the key optimizations is the texture pool system, which dramatically reduces memory allocation overhead:
class TexturePool {
private var textureCache: [TextureKey: [MTLTexture]] = [:]
func getTexture(width: Int, height: Int, pixelFormat: MTLPixelFormat) -> MTLTexture {
// Check cache first
// Create new texture if needed
// Return texture for use
}
func returnTexture(_ texture: MTLTexture) {
// Return texture to cache for reuse
}
}
Real-time Performance Monitoring
Harbeth includes a built-in performance monitoring system that helps identify bottlenecks:
// Enable performance monitoring
Device.setEnablePerformanceMonitor(true)
// Process with HarbethIO
let io = HarbethIO(element: image, filters: filters)
let result = try? io.output()
// Get performance statistics
let stats = PerformanceMonitor.shared.getStatistics()
print("Total Time: \(stats.totalTime)ms | GPU Time: \(stats.gpuTime)ms")
🚀 Advanced Usage Examples
Creating Custom Filters
Harbeth makes it easy to create custom filters using Metal shaders:
class C7CustomFilter: C7FilterProtocol {
var modifier: Modifier { return .compute(kernel: "customKernel") }
// Custom parameters
var intensity: Float = 0.5
func setupSpecialEncodeCommand(with encoder: MTLComputeCommandEncoder) {
encoder.setBytes(&intensity, length: MemoryLayout<Float>.stride, index: 0)
}
}
// Metal shader (customKernel.metal)
/*
kernel void customKernel(
texture2d<half, access::read> input [[texture(0)]],
texture2d<half, access::write> output [[texture(1)]],
constant float &intensity [[buffer(0)]],
uint2 gid [[thread_position_in_grid]]
) {
half4 color = input.read(gid);
// Custom processing
color.rgb = mix(color.rgb, half3(1.0), intensity);
output.write(color, gid);
}
*/
Complex Filter Chains
Combine multiple filters for sophisticated effects:
// Create a cinematic effect chain
let cinematicFilter = C7CombinationCinematic()
// Or build your own custom chain
let filters: [C7FilterProtocol] = [
C7Brightness(brightness: 0.1),
C7Contrast(contrast: 1.2),
C7Saturation(saturation: 1.1),
C7Vignette(intensity: 0.5),
C7Granularity(grain: 0.3)
]
// Apply with operator chaining
let result = image ->> filters
Real-time Camera Processing
Process camera feed with filters at 60 FPS:
// Setup camera collector
let camera = C7CollectorCamera(delegate: self)
camera.captureSession.sessionPreset = .hd1280x720
// Apply real-time filters
let filters = [C7EdgeGlow(lineColor: .red)]
camera.filters = filters
// Start capture
camera.startRunning()
// Handle filtered frames
extension CameraViewController: C7CollectorImageDelegate {
func preview(_ collector: C7Collector, fliter image: C7Image) {
// Update UI with filtered image
DispatchQueue.main.async {
self.imageView.image = image
}
}
}
📊 Performance Breakdown
Key Performance Insights
- GPU Acceleration: Up to 5x faster than CPU processing
- Scalability: GPU time grows logarithmically while CPU time grows linearly
- Memory Efficiency: Texture pooling reduces memory usage by up to 70%
- Real-time Capability: Maintain 60 FPS even with 10+ filters
💻 macOS-specific Features
Harbeth includes several macOS-specific optimizations:
- AppKit Integration: Seamless integration with NSImage and AppKit
- High-Resolution Support: Optimized for Retina displays
- Multi-window Processing: Handle multiple images across different windows
- Drag & Drop Support: Easy integration with macOS drag and drop
// macOS-specific implementation
class MacImageProcessor {
func processDroppedImage(_ image: NSImage) {
let filters = [C7ColorMatrix4x4(matrix: .sepia)]
let result = try? image.make(filters: filters)
// Update UI with result
}
}
🎨 SwiftUI Integration
Harbeth provides native SwiftUI support with the HarbethView component:
import SwiftUI
import Harbeth
struct FilteredImageView: View {
@State private var inputImage: UIImage = UIImage(named: "sample")!
@State private var intensity: Float = 0.5
var body: some View {
VStack {
HarbethView(image: inputImage, filters: [
CIHighlight(highlight: intensity),
C7WaterRipple(ripple: intensity),
]) { image in
image
.resizable()
.aspectRatio(contentMode: .fit)
.cornerRadius(12)
.shadow(radius: 5)
}
Slider(value: $intensity, in: 0...1)
.padding()
}
.padding()
}
}
🌟 Use Cases and Success Stories
Camera Apps
- Real-time filters during capture
- Beauty effects with skin smoothing
- Creative filters for social media
Photo Editors
- Professional color grading
- Batch processing of multiple images
- Artistic effects and stylization
Video Processing
- Filtered video playback
- Real-time effects during recording
- Video enhancement and color correction
📚 Getting Started
Installation
CocoaPods:
pod 'Harbeth'
Swift Package Manager:
dependencies: [
.package(url: "https://github.com/yangKJ/Harbeth.git", branch: "master"),
]
Quick Start
import Harbeth
// Method 1: HarbethIO
let io = HarbethIO(element: image, filters: [C7SoulOut(soul: 0.7)])
let result = try? io.output()
// Method 2: Direct extension
let result = try? image.make(filters: [C7SoulOut(soul: 0.7)])
// Method 3: Operator chaining
let result = image ->> C7SoulOut(soul: 0.7)
🤔 Common Questions
Q: How does Harbeth compare to CoreImage?
A: Harbeth offers better performance for complex filter chains and a more intuitive API, while still integrating with CoreImage for additional functionality.
Q: Can I use Harbeth in production?
A: Yes, Harbeth is production-ready and used in several apps. It's stable, well-documented, and actively maintained.
Q: How do I handle memory constraints?
A: Harbeth includes automatic texture pooling and memory management. For memory-constrained devices, use Device.setMemoryLimitMB() to set appropriate limits.
Q: Does Harbeth support live camera processing?
A: Yes, Harbeth provides C7CollectorCamera for real-time camera processing with filters applied at 60 FPS.
📞 Support and Community
- GitHub Repository: https://github.com/yangKJ/Harbeth
- GitHub Discussions: Discuss Harbeth
- Issues: Bug reports and feature requests
🎯 Future Roadmap
- More filters and effects
- Enhanced Metal Shader support
- Machine learning integration for advanced effects
- Improved documentation and examples
- Community contributions and plugin system
Have you used Harbeth in your projects? I'd love to hear about your experiences and see what creative effects you've created. Let's push the boundaries of image processing together!
Top comments (0)