Local Camera Analysis with Frigate NVR and Coral TPU

Real-time analysis of video streams from IP cameras, especially for object detection, requires intensive processing power. To achieve this processing power on your local network with low latency and while preserving your privacy, without relying on cloud services, Frigate NVR and Google Coral TPU form a powerful duo. This combination offers an ideal solution, especially in homelab environments or small businesses, for detecting objects like people and vehicles from camera footage to instantly record events and trigger automations.

This guide provides a step-by-step roadmap for installing Frigate NVR on Docker with Coral TPU support and performing the basic configuration. Our goal is to eliminate cloud dependency, thereby increasing data security and creating an instantly responsive, efficient security and automation system. Local processing ensures the system continues to operate even if the internet connection is lost and saves on subscription fees.

What is Frigate NVR and Why Local Analysis?

Frigate NVR is an open-source, lightweight, and real-time object detection-capable network video recorder (NVR) solution. Its primary function is to process video streams from IP cameras, detect specified objects (e.g., people, cars), and record these events. Frigate performs these detections using an AI model and publishes the results via MQTT, facilitating integration with other systems.

Local analysis is one of Frigate's biggest advantages. The fact that image processing and object detection operations occur entirely on your own server ensures that your sensitive data is not sent to third-party cloud providers. This is especially important in scenarios where privacy is critical, such as home or business security. Furthermore, unlike cloud services, local analysis does not require any subscription fees and continues to function even if the internet connection is lost. This increases the reliability of the system and provides a long-term cost advantage.

ℹ️ Alternative to Cloud-Based Solutions

While cloud-based camera systems often offer easy setup, they export your data and come with monthly subscription fees. Local solutions like Frigate, despite the initial setup effort, provide more control, privacy, and cost-effectiveness in the long run. Especially if you have multiple cameras, the return on a local solution increases even further.

Another benefit of local processing is low latency. The time between object detection and recording trigger is minimized because there's no need to upload and process the image on a server. This allows for instant reactions in automation scenarios, such as turning on lights or sending a notification when motion is detected. Thanks to Frigate's MQTT integration, you can easily communicate with smart home platforms like Home Assistant and create complex automations based on detected events.

Google Coral TPU: The Power of Edge AI

The Google Coral TPU (Tensor Processing Unit) is specialized hardware designed to accelerate artificial intelligence inference operations. It can run TensorFlow Lite models extremely efficiently, providing a significant performance boost in tasks like real-time object detection by offloading the heavy burden from the CPU. In systems like Frigate NVR, using a Coral TPU enables the ability to analyze multiple video streams simultaneously with low latency.

The importance of the Coral TPU becomes apparent, especially when processing multiple high-resolution camera streams or on low-power hardware. CPU-based object detection often comes with high resource consumption, which can degrade overall system performance and even lead to freezes. The Coral TPU handles this computational intensity, freeing up the main processor for other tasks. This allows you to manage multiple cameras smoothly even with a single mini PC or an affordable device like a Raspberry Pi.

Various Coral TPU form factors are available on the market, such as the USB Accelerator and PCIe card. The USB Accelerator is a popular choice due to its compatibility with most devices and its portability. The PCIe card, on the other hand, is designed for internal use in server-type systems or mini PCs. Your choice will depend on your existing hardware's expansion slots and the performance you need. In both cases, Frigate works directly with the Coral TPU for maximum efficiency.

💡 Is One Coral Enough?

For most home or small office environments, a single Coral TPU USB Accelerator is sufficient. Frigate can efficiently process streams from multiple cameras on a single TPU. However, if you have a large number of high-resolution cameras or want to perform analysis at very high FPS values, you might consider multiple Coral TPUs or a more powerful model.

Pre-Installation Preparations and Required Components

Before starting to set up the local camera analysis system with Frigate NVR and Coral TPU, some hardware and software preparations are necessary. Selecting the right components and making preliminary settings will ensure a smooth installation process.

Hardware Requirements

1. Server: You need a host computer to run Frigate.
   • Mini PC (Intel NUC, Beelink, etc.) or an old PC: x86-based processors generally offer better performance and are ideal for running multiple Docker containers. Ample storage space and sufficient RAM (at least 8GB recommended) are important.
   • Raspberry Pi 4 (4GB or 8GB RAM): A popular option for its low power consumption and compact size. However, it may reach performance limits for a large number of cameras or high-resolution streams. Frigate performs best on bare metal Debian-based distributions or with Docker on Raspberry Pi OS.
   • Storage: Sufficient space on an SSD or HDD is required for camera recordings. NVMe SSDs or high-endurance HDDs are preferable due to continuous recording writes.
2. IP Cameras: You need IP cameras that support RTSP (Real-Time Streaming Protocol). Most modern security cameras offer this feature.
3. Google Coral TPU: An essential component to boost object detection performance.
   • Coral USB Accelerator: The most common and affordable option. Requires a free USB 3.0 port on your server.
   • Coral M.2 Accelerator or PCIe Accelerator: More for internal use, suitable for servers with appropriate M.2 or PCIe slots.

Software Requirements

1. Operating System: A Linux-based operating system is recommended. Distributions like Ubuntu Server, Debian, or Fedora CoreOS are good choices. The operating system should be up-to-date and stable. Frigate performs best on bare metal Debian-based distributions.
2. Docker and Docker Compose: Frigate runs as a Docker container. Docker and Docker Compose must be installed on your server. Refer to Docker's official documentation for installation instructions.
3. MQTT Broker (optional but recommended): Frigate publishes detected events via MQTT. Installing an MQTT broker (e.g., Mosquitto) is necessary to integrate Frigate with other smart home systems like Home Assistant. Eclipse Mosquitto is a popular choice.

Network and Other Preparations

• Static IP Addresses: Assigning static IP addresses to your cameras and Frigate server makes network configuration more stable. This can be done by reserving IPs in your DHCP server or by directly configuring static IPs on the cameras.
• RTSP Stream Information: Note down the RTSP URL (including username, password, and port) and possibly the low-resolution sub-stream URL for each camera. This information will be used in the frigate.yml configuration.
• Storage Path: Determine an appropriate directory structure on your server for recordings and configuration files. For example, paths like /opt/frigate/config and /opt/frigate/recordings are used during Docker Compose volume mapping.

ℹ️ Hardware Selection and Commercial Decisions

When designing a system architecture in a production ERP or a customer project, hardware and software choices are always a matter of trade-offs. The same applies to Frigate and Coral. A low-cost Raspberry Pi 4 might be attractive in terms of power consumption and initial cost, but an older mini PC or NUC could be a better investment for long-term performance and expandability. Especially if you have multiple cameras and high-resolution streams, opting for an x86-based system with a more powerful CPU and RAM prevents potential bottlenecks in the future. Remember, hardware lifespan and durability are also important factors, especially for an NVR system where recordings are continuously written.

Once these preparations are complete, you will be ready to proceed with the Frigate and Coral TPU installation steps.

Frigate and Coral TPU Installation Steps

Installing Frigate NVR with Docker Compose simplifies management and isolates dependencies. In this section, we will cover the basic Docker Compose file and an example Frigate configuration.

Basic Installation with Docker Compose

Creating a docker-compose.yml file to run Frigate and an MQTT broker (Mosquitto) simultaneously is the most practical way. The example below includes the necessary services and configurations for a basic setup.

# /opt/frigate/docker-compose.yml
version: "3.9"
services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true # May be required for USB Coral access, but more secure methods should be preferred
    ports:
      - "5000:5000" # Frigate web UI
      - "1935:1935" # For RTMP streams (optional)
    volumes:
      - /dev/bus/usb:/dev/bus/usb # For Coral USB Accelerator
      - /opt/frigate/config:/config # Frigate configuration file
      - /opt/frigate/recordings:/media/frigate/recordings # Where recordings are stored
      - /etc/localtime:/etc/localtime:ro # For correct timezone
    environment:
      - TZ=Europe/Istanbul # Set your timezone
    devices:
      - /dev/apex_0:/dev/apex_0 # For Coral PCI/USB device (may vary by device)
    image: blakeblackshear/frigate:stable # Current stable Frigate version
    depends_on:
      - mqtt # Dependency on MQTT service
  mqtt:
    container_name: mqtt
    restart: unless-stopped
    image: eclipse-mosquitto:latest # Mosquitto MQTT broker
    ports:
      - "1883:1883" # MQTT default port
    volumes:
      - /opt/frigate/mqtt/config:/mosquitto/config # MQTT configuration file
      - /opt/frigate/mqtt/data:/mosquitto/data # MQTT persistent data

After creating this docker-compose.yml file in your frigate directory, you may need to create the corresponding directories (/opt/frigate/config, /opt/frigate/recordings, /opt/frigate/mqtt/config, /opt/frigate/mqtt/data) on your server and assign appropriate permissions. Specifically, the /dev/bus/usb and /dev/apex_0 paths may vary depending on your Coral TPU type and how your system recognizes it. You can check your USB devices with the lsusb command and your Coral TPU device (if present) with the ls /dev/apex* command.

The privileged: true setting allows the Docker container direct access to devices on the host system and may sometimes be necessary for accessing hardware devices like the Coral TPU. However, this setting carries security risks because it grants the container access to all devices on the host system. If possible, it is safer to specify only the necessary devices with the devices key and disable privileged mode.

Frigate Configuration File (frigate.yml)

You need to create a frigate.yml file in the /opt/frigate/config directory for the frigate service. This file defines your cameras, detection settings, and other Frigate features.

# /opt/frigate/config/frigate.yml
mqtt:
  host: mqtt # Name of the MQTT service within Docker Compose
  topic_prefix: frigate # Prefix for MQTT messages
  user: your_mqtt_user # Your MQTT username (if any)
  password: your_mqtt_password # Your MQTT password (if any)

detectors:
  coral:
    type: edgetpu
    device: usb # Or pci (if using a PCIe card)

cameras:
  on_site_camera_1: # A unique name for your camera
    ffmpeg:
      inputs:
        - path: rtsp://user:password@192.168.1.100:554/stream1 # Main stream, high resolution
          roles:
            - detect # Use this stream for object detection
            - record # Record this stream
        - path: rtsp://user:password@192.168.1.100:554/stream2 # Sub stream, low resolution
          roles:
            - rtmp # For live viewing in the web interface
            - tiny_motion # For motion detection (optional)
      # hwaccel_args: -c:v h264_qsv # For hardware acceleration (depends on system support)
    detect:
      enabled: True
      threshold: 0.7 # Detection threshold (0.6-0.8 gives good results)
      stationary_threshold: 10 # Determines how long an object must remain stationary
      min_area: 5000 # Minimum pixel area of the object to be detected
      max_area: 1000000 # Maximum pixel area of the object to be detected
      objects:
        - person
        - car
      zones: # Detection zones (optional)
        main_entrance:
          coordinates:
            - 0,0
            - 640,0
            - 640,480
            - 0,480
          objects: # Only detect specific objects in this zone
            - person
    record:
      enabled: True
      retain:
        days: 7 # How many days to retain recordings
        mode: all # or motion (only record when there is motion)
    rtmp:
      enabled: True # Enable RTMP stream for live viewing

In the frigate.yml file, correctly specifying the RTSP stream paths for your cameras in the ffmpeg section is critical. The roles setting determines the purpose of each stream. Typically, the high-resolution main stream is used for detect and record, while the low-resolution sub-stream takes on the rtmp role (for live viewing) or tiny_motion (for simple motion detection on the CPU). This distinction helps optimize resource usage. With zones and objects definitions, you can reduce false notifications by fine-tuning detection sensitivity and specifying which objects are of interest to you.

Starting the Installation

After creating the docker-compose.yml and frigate.yml files, navigate to the /opt/frigate directory in your terminal and start the Frigate and MQTT services with the following command:

cd /opt/frigate
docker compose up -d

Once the services have started, you can access the Frigate web interface at http://<Frigate_Server_IP_Address>:5000. Here you can view live camera streams, monitor detected events, and check the system status.

⚠️ Initial Startup Issues

When setting up a new system, especially concerning hardware and software integration, it's common to encounter some issues initially. Checking the Frigate container logs with the docker logs frigate command is the best way to identify potential errors and configuration problems. The most common issues include incorrect RTSP URLs, problems accessing the Coral TPU (permissions or wrong /dev path), and MQTT connection errors. These logs will guide you in finding the root cause of the problem.

Performance Settings and Optimizations

Proper performance settings and optimization of system resources are crucial for Frigate NVR to operate efficiently. These optimizations become critical, especially when working with multiple cameras or high-resolution streams.

Video Stream Management

The most significant factor affecting Frigate's performance is how camera streams are processed. Most IP cameras offer a high-resolution main stream and a low-resolution sub-stream. Using these two streams correctly significantly reduces the load on the CPU and Coral TPU:

• Low-Resolution detect Stream: Always use the low-resolution stream of your cameras for object detection (detect role). For example, resolutions like 640x480 or 1280x720 provide sufficient detail for object detection and allow the Coral TPU to perform inference faster. Using high-resolution 4K or 1080p streams directly for detect can strain your Coral TPU and reduce frame rates.
• High-Resolution record Stream: For recordings (record role), using the high-resolution main stream allows you to obtain detailed images without sacrificing video quality.
• rtmp Stream: For live viewing in Frigate's web interface, a low-resolution stream (rtmp role) is generally preferred. This means faster loading times in the browser and less network bandwidth consumption.

Enabling hardware acceleration using the hwaccel_args parameter under ffmpeg in the frigate.yml file can significantly reduce CPU usage. Options like h264_qsv for Intel processors, h264_nvenc for NVIDIA GPUs, or h264_v4l2m2m for devices like Raspberry Pi are available. However, this requires your system and Docker installation to support hardware acceleration and can make the setup slightly more complex.

Detection Parameters and Zones

Fine-tuning the parameters in Frigate's detect section is critical for reducing false positives and ensuring the system focuses only on important events:

• zones: Performing object detection only in specific areas of your cameras' view both prevents unnecessary detections and improves performance. For example, you can have a camera monitor only a doorway or a parking area. Zones can also be associated with specific objects (objects list) to be detected.
• objects: Specifying which objects Frigate should detect (e.g., only person and car) eliminates unnecessary workload. By default, Frigate can detect many objects from the COCO dataset, but for most users, only a few are relevant.
• threshold: Determines the confidence threshold required for an object to be detected. A higher threshold means fewer false positives (e.g., tree shadows being detected as people) but potentially more missed detections. Generally, 0.6 to 0.8 is a good starting point.
• stationary_threshold: Determines how long an object must remain stationary. This is useful for preventing false triggers caused by objects swaying in the wind.

Coral TPU Usage

A single Coral TPU provides sufficient performance for most homelab scenarios. Frigate efficiently queues and processes multiple camera streams on a single TPU. However, when working with very high FPS values (e.g., above 10 FPS for each camera) and a large number of high-resolution streams, attention must be paid to the Coral TPU's capacity. You can monitor the Coral TPU's utilization rate from the "System" section of the Frigate web interface. If it's consistently running near 100%, you may need to reduce the number of streams, resolution, or FPS.

⚠️ Disk Usage and Management

Camera recordings, especially high-resolution and continuous recordings, quickly consume disk space. You can determine how many days recordings are retained with the record.retain.days setting in frigate.yml. This is a critical setting to prevent your disk from filling up. Also, ensure that the directory where Frigate stores recordings (/opt/frigate/recordings) has sufficient capacity and is on a performant storage device. Continuous write operations are not suitable for low-quality SD cards or USB drives and can shorten their lifespan.

These optimizations will ensure your Frigate NVR system runs stably, quickly, and resource-efficiently. Don't forget to restart the Docker container (docker compose restart frigate) after making configuration changes.

Security and Privacy Tips

While setting up a local NVR system offers privacy advantages, ensuring the security of the system itself remains paramount. Camera streams and recorded data on your network can contain sensitive information, so security measures must be taken seriously.

Network Isolation and Segmentation

Isolating your Frigate server and IP cameras from the rest of your network reduces potential security risks. I have observed many clients' network structures where cameras are placed on a separate VLAN, such as an IoT or guest network. This prevents a potential vulnerability originating from cameras (e.g., a bug in camera firmware) from spreading to your main network.

🔥 Network Security Measures

Moving your cameras and Frigate server to a separate VLAN, and restricting traffic from this VLAN to the main network with Access Control Lists (ACLs) or firewall rules, significantly narrows the potential attack surface. Only allow the Frigate server to access cameras via the RTSP port (usually 554), and if necessary, allow access from the main network for the Frigate web interface (port 5000).

Camera Access and Credentials

Changing the default usernames and passwords of your IP cameras is the first and most fundamental security step. Use strong, unique passwords for RTSP streams. If your cameras support features like UPnP (Universal Plug and Play), you might consider disabling them due to security risks. Keeping your camera firmware up-to-date also helps protect against known security vulnerabilities.

Frigate Web Interface Security

Frigate's web interface is accessible by default without authentication. While this may not be an issue on your internal network, if you want to provide external access, you must add additional security layers:

1. Reverse Proxy (Nginx/Caddy): Instead of direct port forwarding to the Frigate interface, set up an Nginx or Caddy reverse proxy to provide access. This allows you to add HTTPS encryption, a basic authentication layer, and even measures like rate limiting.
2. HTTPS: Obtain a free SSL certificate from a service like Let's Encrypt via your reverse proxy to provide encrypted access to the Frigate interface.
3. Authentication: You can integrate HTTP Basic Authentication or a more advanced authentication solution on Nginx or Caddy.
4. fail2ban: You can use fail2ban to monitor failed login attempts to the Frigate server via SSH or the web interface and automatically block malicious IP addresses.
5. Security Updates: Regularly updating the Frigate Docker image and the underlying operating system helps protect against known security vulnerabilities.

Data Privacy and Storage

Recorded video footage can contain sensitive personal data. Ensure these recordings are stored securely:

• Local Storage: Since Frigate keeps recordings on your local server, you are free from cloud dependency. However, this means you need to secure your local storage.
• Disk Encryption: Encrypting the disk where recordings are stored (e.g., with LUKS) prevents unauthorized access to your data even if the server is physically stolen.
• Access Control: Restrict users and systems that have access to Frigate recordings. If necessary, tighten file system permissions (chmod, chown) for recording directories.

By implementing these security and privacy tips, you can ensure your Frigate NVR system operates both performantly and securely. Security is not a one-time task but an ongoing process.

Conclusion

The combination of Frigate NVR and Google Coral TPU offers a powerful, privacy-focused, and low-latency camera analysis solution on your local network. With the installation steps and optimization strategies covered in this guide, you can build your own security and automation system without relying on cloud-based systems. Keeping your data under your control, creating instantly responsive automations, and eliminating subscription costs are among the greatest benefits of this approach.

Remember, good system performance and security are achieved not only with initial setup but also with continuous monitoring, regular updates, and fine-tuning. Frigate's open-source community and flexible configuration options allow you to continuously develop the system according to your needs. Now that your smart NVR is up and running, the next step might be to integrate it with smart home platforms like Home Assistant to design more complex automation scenarios based on detected events.

Official Resources

• oneuptime.com
• wordpress.com
• thepihut.com
• mostlychris.com
• frigate.video
• frigate.video
• github.com

- artifacthub.io