Building an ESP32-C3 WiFi MQTT Client for IoT Data Streaming (DevKitM-1 / Rust-1)

Building an ESP32-C3 WiFi MQTT Client for IoT Data Streaming (DevKitM-1 / Rust-1)

The Problem: Connecting ESP32 Devices to the Cloud

When building IoT projects with ESP32 devices, one of the biggest challenges is establishing reliable cloud connectivity. Hardcoded MQTT broker settings create several problems:

• Deployment Complexity: Different environments need different MQTT brokers
• Security Risk: Broker credentials are visible in source code
• Maintenance Nightmare: Updating broker settings requires recompiling and reflashing
• No Real-time Monitoring: Difficult to verify data is reaching the cloud

A Practical Solution: Externalized MQTT Configuration with Real-time Monitoring

Here's a comprehensive approach that extends WiFi connectivity with MQTT cloud streaming while keeping the code clean and maintainable:

🔐 Externalized Configuration Management

• WiFi and MQTT credentials stored in separate files
• Template files for easy setup and deployment
• Git-ignored credentials to prevent accidental commits
• No hardcoded passwords or broker settings in source code

📡 Real-time Data Streaming

• Device sensor data published every 30 seconds
• Heartbeat messages every 60 seconds for health monitoring
• JSON-formatted messages for easy cloud processing
• Command handling for remote device control

💡 Visual Status Indicators

• Built-in LED shows WiFi and MQTT connection status
• Fast blink = Strong signal + MQTT connected
• Medium blink = Medium signal + MQTT connected
• Slow blink = Weak signal + MQTT connected
• LED off = No WiFi connection

🔄 Robust Connection Management

• Auto-reconnection for both WiFi and MQTT
• Independent monitoring every 5-10 seconds
• Graceful handling of network failures
• Comprehensive error logging

🛠️ Arduino IDE Ready

• Works seamlessly with Arduino IDE
• Simple library management (PubSubClient, ArduinoJson)
• Easy setup scripts for Windows and Linux/Mac
• No complex build systems or CLI tools

Code Architecture

Main Sketch Structure

##include <WiFi.h>
##include <PubSubClient.h>
##include <ArduinoJson.h>
##include "wifi_config.h"    // WiFi credentials (gitignored)
##include "mqtt_config.h"    // MQTT broker settings (gitignored)

##define LED_PIN 7 // ESP32C3 built-in LED

// Publishing intervals
const unsigned long DATA_PUBLISH_INTERVAL = 30000; // 30 seconds
const unsigned long HEARTBEAT_INTERVAL = 60000;    // 60 seconds

void setup() {
  // Initialize serial, LED, WiFi, and MQTT
  connectToWiFi();
  setupMQTT();
}

void loop() {
  // Monitor connections and publish data
  checkWiFiConnection();
  checkMQTTConnection();

  if (wifiConnected && mqttConnected) {
    publishDeviceData();
    publishHeartbeat();
  }

  updateLEDStatus();
  mqttClient.loop();
}

Configuration Management

// wifi_config.h (gitignored)
const char* WIFI_SSID = "YourWiFiNetwork";
const char* WIFI_PASSWORD = "YourWiFiPassword";

// mqtt_config.h (gitignored)
const char* MQTT_BROKER = "broker.hivemq.com";
const int MQTT_PORT = 1883;
const char* MQTT_CLIENT_ID = "ESP32_Client";
const char* MQTT_TOPIC_PREFIX = "esp32/device";

Key Features Explained

1. Real-time Data Publishing

The device publishes comprehensive sensor data every 30 seconds:

void publishDeviceData() {
  DynamicJsonDocument doc(512);
  doc["device_id"] = MQTT_CLIENT_ID;
  doc["timestamp"] = millis();
  doc["wifi_rssi"] = WiFi.RSSI();
  doc["wifi_ssid"] = WiFi.SSID();
  doc["free_heap"] = ESP.getFreeHeap();
  doc["uptime"] = millis() / 1000;
  doc["mqtt_connected"] = mqttConnected;

  String topic = String(MQTT_TOPIC_PREFIX) + "/data";
  String payload;
  serializeJson(doc, payload);

  mqttClient.publish(topic.c_str(), payload.c_str());
}

2. Health Monitoring with Heartbeat

Regular heartbeat messages ensure device health monitoring:

void publishHeartbeat() {
  DynamicJsonDocument doc(256);
  doc["device_id"] = MQTT_CLIENT_ID;
  doc["timestamp"] = millis();
  doc["status"] = "alive";
  doc["wifi_connected"] = wifiConnected;
  doc["mqtt_connected"] = mqttConnected;

  String topic = String(MQTT_TOPIC_PREFIX) + "/heartbeat";
  String payload;
  serializeJson(doc, payload);

  mqttClient.publish(topic.c_str(), payload.c_str());
}

3. Remote Device Control

The device responds to MQTT commands for remote control:

void handleCommand(JsonDocument &doc) {
  if (doc.containsKey("led")) {
    String ledCommand = doc["led"];
    if (ledCommand == "on") {
      digitalWrite(LED_PIN, HIGH);
      Serial.println("LED turned ON via MQTT");
    } else if (ledCommand == "off") {
      digitalWrite(LED_PIN, LOW);
      Serial.println("LED turned OFF via MQTT");
    }

    // Send acknowledgment
    publishCommandResponse("led", ledCommand, "success");
  }
}

4. Smart LED Status System

The LED provides instant visual feedback about connection status:

void updateLEDStatus() {
  if (!wifiConnected) {
    // No WiFi - LED off
    digitalWrite(LED_PIN, LOW);
  } else if (!mqttConnected) {
    // WiFi connected, no MQTT - slow blink
    ledState = !ledState;
    digitalWrite(LED_PIN, ledState);
    ledBlinkInterval = 1000; // 1 second
  } else {
    // Both connected - show signal strength
    int rssi = WiFi.RSSI();
    if (rssi > -50) {
      // Strong signal - fast blink
      ledBlinkInterval = 100; // 100ms
    } else if (rssi > -70) {
      // Medium signal - medium blink
      ledBlinkInterval = 500; // 500ms
    } else {
      // Weak signal - slow blink
      ledBlinkInterval = 1000; // 1 second
    }
  }
}

MQTT Message Types and Frequency

Published Messages

Message Type	Frequency	Topic	Purpose
Device Data	Every 30 seconds	esp32/device/data	Sensor data and status
Heartbeat	Every 60 seconds	esp32/device/heartbeat	Health monitoring
Connection Status	On connect	esp32/device/status	Connection confirmation
Command Response	Per command	esp32/device/response	Command acknowledgments

Subscribed Messages

Message Type	Topic	Purpose
Commands	esp32/device/commands	Remote device control
Configuration	esp32/device/config	Settings updates

Setup Instructions

1. Hardware Requirements

• ESP32 C3 DevKitM1 board
• USB cable for programming and power
• WiFi network with internet access
• MQTT broker (public test broker included)

2. Software Setup

1. Install Arduino IDE
2. Add ESP32 board package URL:

   https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json

1. Install "ESP32 by Espressif Systems" from Boards Manager
2. Install required libraries:
   • PubSubClient by Nick O'Leary
   • ArduinoJson by Benoit Blanchon

3. Project Configuration

1. Run setup script: ./setup.sh (Linux/Mac) or setup.bat (Windows)
2. Edit wifi_config.h with your WiFi credentials
3. Edit mqtt_config.h with your MQTT broker settings
4. Select board: "ESP32C3 Dev Module"
5. Upload and monitor at 115200 baud

4. Testing MQTT Messages

Use web-based MQTT clients to monitor messages:

• HiveMQ WebSocket Client: https://www.hivemq.com/demos/websocket-client/
• MQTT Dashboard: https://mqtt-dashboard.com/
• Subscribe to: esp32/device/#

5. Expected Serial Output

When everything is working correctly, you'll see output like this:

ESP32C3 WiFi Connection
Connecting to WiFi...
SSID: RWN
...
WiFi Connected!
IP: 192.168.12.11
RSSI: -51 dBm
Setup complete!
MQTT disconnected! Attempting to reconnect...
Connecting to MQTT broker: test.mosquitto.org
MQTT connected!
Subscribed to: esp32/device/commands
Subscribed to: esp32/device/config
✓ Data published to: esp32/device/data
  Payload: {"device_id":"ESP32_Client","timestamp":30032,"wifi_rssi":-52,"wifi_ssid":"RWN","free_heap":201648,"uptime":30,"mqtt_connected":true}
✓ Heartbeat published to: esp32/device/heartbeat
✓ Data published to: esp32/device/data
  Payload: {"device_id":"ESP32_Client","timestamp":60118,"wifi_rssi":-48,"wifi_ssid":"RWN","free_heap":199964,"uptime":60,"mqtt_connected":true}

Security Considerations

✅ Improvements Over Hardcoded Settings

• Credentials are externalized from source code
• Template files provide clear setup instructions
• Git ignores actual credentials
• No secrets in version control
• MQTT messages contain no sensitive data

⚠️ Security Limitations

• Credentials are still stored in plain text
• No encryption at rest
• Public MQTT broker for testing
• Consider this a starting point, not a complete security solution

🔒 For Production Use, Consider:

• AWS IoT Core with X.509 certificates
• Encrypted MQTT connections (MQTTS)
• Device authentication and secure boot
• Private MQTT brokers
• OTA updates for configuration changes

Troubleshooting Common Issues

MQTT Connection Fails

• Verify MQTT broker settings in mqtt_config.h
• Check internet connectivity
• Try different MQTT broker (test.mosquitto.org)
• Verify broker port (1883 for standard MQTT)

No Messages in MQTT Client

• Check ESP32 Serial Monitor for "MQTT connected!" message
• Verify topic subscription: esp32/device/#
• Ensure both WiFi and MQTT are connected
• Check LED status indicators

WiFi Connection Fails

• Verify credentials in wifi_config.h
• Check WiFi network availability
• Ensure 2.4GHz network (ESP32 doesn't support 5GHz)

Board Not Found

• Check USB connection and drivers
• Verify COM port selection
• Try different USB cable

Project Structure

esp32-wifi-mqtt-cpp/
├── esp32-wifi-mqtt/
│   ├── esp32-wifi-mqtt.ino        # Main Arduino sketch
│   ├── wifi_config_template.h     # WiFi credentials template
│   ├── mqtt_config_template.h     # MQTT configuration template
│   ├── setup.sh                   # Linux/Mac setup script
│   └── setup.bat                  # Windows setup script
├── .gitignore                     # Git ignore rules
├── LICENSE                        # MIT License
└── README.md                      # Comprehensive documentation

Why This Approach Works

For Development

• Clean separation of concerns
• Easy to test with different brokers
• No accidental credential commits
• Clear project structure
• Real-time message monitoring

For Field Deployment

• Simple setup process
• Visual feedback for troubleshooting
• Robust connection handling
• Easy configuration updates
• Comprehensive error logging

For Production Scaling

• Foundation for AWS IoT Core integration
• Terraform infrastructure automation
• Fleet management capabilities
• Cloud data processing pipeline
• Enterprise-grade security

Next Steps

This foundation can be extended with:

• AWS IoT Core Integration: Enterprise cloud connectivity
• Terraform Infrastructure: Automated cloud deployment
• Fleet Management: Scale to hundreds of devices
• Data Analytics: Cloud-based data processing
• OTA Updates: Remote firmware updates
• Advanced Security: Certificate-based authentication

Conclusion

Building reliable IoT cloud connectivity doesn't have to be complicated. This ESP32 WiFi MQTT client demonstrates how to:

• Stream real-time data to the cloud
• Provide visual feedback for troubleshooting
• Handle connection issues gracefully
• Maintain clean, readable code
• Deploy easily in the field
• Scale to production environments

This approach provides a solid foundation for IoT projects that need reliable cloud connectivity without complex infrastructure requirements.

Ready to connect your ESP32 to the cloud? Start with externalized configuration and real-time monitoring. It's a small change that makes a big difference in reliability and maintainability!

📁 Full Project Details

For the complete source code, setup instructions, and examples, check out the full project:

ESP32 WiFi MQTT Client Project

The repository includes:

• Complete Arduino sketch with WiFi and MQTT functionality
• Template files for easy setup
• Comprehensive README with troubleshooting
• Setup scripts for Windows and Linux/Mac
• AWS IoT Core integration roadmap
• Terraform infrastructure automation

---

What's your experience with IoT cloud connectivity? Have you found better approaches for MQTT integration? Let me know in the comments!

IoT #ESP32 #Arduino #MQTT #WiFi #CloudConnectivity #EmbeddedSystems #C++ #DataStreaming #AWS