DEV Community

Moisi Trungu
Moisi Trungu

Posted on • Originally published at flixzone.icu

Asterisk 'All Circuits Busy' Error — SIP Channel Exhaustion Fix

#devops #linux #sip #voip

Asterisk 'All Circuits Busy' Error — SIP Channel Exhaustion Fix

Learn how to diagnose, prevent, and resolve the "all circuits busy" error in production Asterisk/ViciDial systems by managing SIP channel limits, monitoring active sessions, and implementing proper resource constraints.

Prerequisites

  • Running Asterisk 11+ with ViciDial 2.14+
  • Root or sudo access to the Asterisk server
  • Basic familiarity with Asterisk CLI and config file editing
  • Access to MySQL/MariaDB and the asterisk database
  • Knowledge of SIP carrier configuration
  • Ability to tail logs and parse Asterisk debugging output

Understanding the "All Circuits Busy" Error

The "All Circuits Busy" error occurs when Asterisk cannot allocate a new SIP channel because:

  1. Channel limits are exhausted — The maxchannels setting for a SIP peer/trunk has been reached
  2. Global SIP limits are hit — The system-wide SIP channel count exceeds configured limits
  3. Memory constraints — Asterisk lacks sufficient resources to create new channels
  4. Resource starvation — CPU, threads, or file descriptors are at capacity
  5. Carrier/trunk limits — Your SIP provider has rate-limited or blocked the connection

In ViciDial environments, this typically manifests as failed outbound calls, dropped inbound calls, or agents seeing a busy signal instead of ring-through.

Section 1: Diagnosing Current Channel Usage

Check Active SIP Channels via CLI

Connect to the Asterisk CLI and run:

asterisk -rx "sip show channels"
Enter fullscreen mode Exit fullscreen mode

This outputs:

Channel              User/ANR            Call ID             Seq (Tx/Rx)  Form
SIP/2001-00000abc    2001               abc123@10.0.0.1      102/101      alaw
SIP/carrier1-00000def 2002               def456@carrier.com   45/44        ulaw
---
SIP/carrier1         carrier1           (none)               0/0          N/A
Total SIP channels: 156/200
Enter fullscreen mode Exit fullscreen mode

The last line shows 156 active channels with a limit of 200. Note the exact number.

Check Peer-Specific Limits

For each carrier or trunk:

asterisk -rx "sip show peer carrier1"
Enter fullscreen mode Exit fullscreen mode

Look for these fields in the output:

Name                 : carrier1
Host                 : 203.0.113.45:5060
Port                 : 5060
Dynamic              : No
Nat                  : No
SendRPID             : No
ACL                  : No
T.38 Support         : No
MaxCallBitrate       : 0 kbps
Channeltype          : SIP
SRTPring cipher      : No
Theoretical          : -1
ChanObjectType       : peer
Language             : en
ChannelDriver        : SIP
Maxchannels          : 50
Current              : 12
Enter fullscreen mode Exit fullscreen mode

The critical field is Maxchannels: 50 with Current: 12. This peer can handle up to 50 simultaneous calls. When Current reaches Maxchannels, new calls fail.

Query ViciDial Database for Call Counts

Connect to MySQL and check active calls:

SELECT COUNT(*) as active_calls 
FROM vicidial_log 
WHERE end_epoch = 0;
Enter fullscreen mode Exit fullscreen mode

Check active closer calls:

SELECT COUNT(*) as active_calls 
FROM vicidial_closer_log 
WHERE end_epoch = 0;
Enter fullscreen mode Exit fullscreen mode

Check carrier usage breakdown:

SELECT carrier_id, COUNT(*) as call_count
FROM vicidial_log 
WHERE end_epoch = 0 
GROUP BY carrier_id 
ORDER BY call_count DESC;
Enter fullscreen mode Exit fullscreen mode

Monitor Asterisk Resource Limits

Check file descriptor limits:

cat /proc/$(pidof asterisk)/limits | grep "open files"
Enter fullscreen mode Exit fullscreen mode

Output example:

Max open files            4096                 4096                 files
Enter fullscreen mode Exit fullscreen mode

For a busy system, 4096 may be insufficient. Each SIP channel requires multiple file descriptors.

Check Asterisk thread count:

ps -eLf | grep asterisk | wc -l
Enter fullscreen mode Exit fullscreen mode

A system with 500+ threads under normal load indicates potential overcommitment.

Check memory usage:

ps aux | grep asterisk | grep -v grep
Enter fullscreen mode Exit fullscreen mode

Output:

asterisk 12345  0.5 18.2 2456780 1523456 ?  Sl  10:15  2:34 /usr/sbin/asterisk -f
Enter fullscreen mode Exit fullscreen mode

The 18.2 is the percentage of system RAM used. Above 25% often indicates problems.

Section 2: Configuring SIP Channel Limits

Global SIP Channel Limits in sip.conf

Edit /etc/asterisk/sip-vicidial.conf:

[general]
; Global settings
port = 5060
bindaddr = 0.0.0.0
context = default
srvlookup = yes

; Channel limits per peer (default if not specified)
maxchannels = 100

; Thread management
; Number of SIP monitor threads
sip_max_threads = 100

; RTP settings
rtpstart = 10000
rtpend = 20000

; Keepalive
keepalive = 60

; TCP/TLS limits
tcpenable = yes
tcpbindaddr = 0.0.0.0
tlsenable = no
maxexpiry = 3600
minexpiry = 60
defaultexpiry = 120
Enter fullscreen mode Exit fullscreen mode

The maxchannels = 100 sets the default per-peer limit. Individual peers override this.

Configure Specific Carrier/Trunk Limits

For each carrier in the [carrier_name] section:

[carrier1]
type = peer
host = 203.0.113.45
port = 5060
username = your_account
secret = your_password
fromuser = your_did
fromdomain = 203.0.113.45
allow = ulaw,alaw,gsm
disallow = all
context = from-carrier
dtmfmode = rfc2833
nat = no
canreinvite = no
qualify = yes
qualifyfreq = 60
maxchannels = 150
Enter fullscreen mode Exit fullscreen mode

Key settings:

  • maxchannels = 150 — This specific peer can have up to 150 simultaneous calls
  • qualify = yes — Periodically test the peer's availability
  • canreinvite = no — Required for ViciDial call recording
  • dtmfmode = rfc2833 — Standard DTFM handling for VOIP

Set Per-Extension Limits for Agent Lines

In /etc/asterisk/extensions-vicidial.conf, limit calls per extension:

[from-internal]
exten => 2001,1,Set(CHANNEL(max_forwards)=10)
exten => 2001,2,Dial(SIP/2001,30,Tt)
exten => 2001,3,Hangup()

exten => 2002,1,Set(CHANNEL(max_forwards)=10)
exten => 2002,2,Dial(SIP/2002,30,Tt)
exten => 2002,3,Hangup()
Enter fullscreen mode Exit fullscreen mode

While max_forwards isn't a channel limit, you can use early media and call state monitoring to prevent dead channels.

Dynamic Peer Limits from ViciDial

ViciDial can dynamically adjust limits via the vicidial_carrier_log table. Check:

SELECT * FROM vicidial_carrier_log 
WHERE carrier_id = 'carrier1' 
ORDER BY date_entered DESC 
LIMIT 1;
Enter fullscreen mode Exit fullscreen mode

ViciDial's quick_closer.pl script reads this table and adjusts Asterisk limits based on carrier performance.

Section 3: Preventing Channel Exhaustion

Implement Call Gating in ViciDial

Edit /usr/share/astguiclient/VICIDIAL_RUN.sh or the campaign settings in the admin panel:

# Check active calls before allowing new dials
SELECT count(*) FROM vicidial_log 
WHERE end_epoch = 0 
AND campaign_id = 'CAMP01';
Enter fullscreen mode Exit fullscreen mode

In the admin panel (/vicidial/admin.php), under Campaign settings, set:

  • Max Simultaneous Calls: 80
  • Max Calls Per Agent: 3
  • Call Drop Rate Threshold: 3% (auto-pause if exceeded)

When active calls reach the limit, the dialer pauses automatically.

Configure Call Timeouts

In /etc/asterisk/extensions-vicidial.conf, set appropriate timeouts:

[from-campaign]
exten => _X.,1,Set(TIMEOUT(absolute)=7200)
exten => _X.,2,Set(TIMEOUT(connect)=45)
exten => _X.,3,Set(TIMEOUT(response)=15)
exten => _X.,4,Dial(SIP/${EXTEN}@carrier1,30,Tt)
exten => _X.,5,Hangup()
Enter fullscreen mode Exit fullscreen mode
  • TIMEOUT(absolute) — Max call duration (7200s = 2 hours)
  • TIMEOUT(connect) — Hangup if no answer after 45 seconds
  • TIMEOUT(response) — Wait 15 seconds for 180 Ringing response

These prevent zombie calls from consuming channels.

Enable SIP Session Timers (RFC 4028)

In sip-vicidial.conf:

[general]
session_timers = accept
session_expires = 600
min_se = 90
Enter fullscreen mode Exit fullscreen mode

This automatically tears down dead SIP sessions after 600 seconds of inactivity, freeing channels.

Monitor and Auto-Restart Asterisk

Create a cron job to check channel health:

#!/bin/bash
# /usr/local/bin/check_asterisk_channels.sh

CHANNEL_COUNT=$(asterisk -rx "sip show channels" | grep "Total" | awk '{print $4}' | cut -d'/' -f1)
MAX_CHANNELS=$(asterisk -rx "sip show channels" | grep "Total" | awk '{print $4}' | cut -d'/' -f2)

THRESHOLD=$(echo "$MAX_CHANNELS * 0.90" | bc | cut -d'.' -f1)

if [ "$CHANNEL_COUNT" -gt "$THRESHOLD" ]; then
    echo "WARNING: Channel count at ${CHANNEL_COUNT}/${MAX_CHANNELS}" | mail -s "Asterisk Alert" admin@example.com

    # Optional: Graceful restart if above 95%
    CRITICAL=$(echo "$MAX_CHANNELS * 0.95" | bc | cut -d'.' -f1)
    if [ "$CHANNEL_COUNT" -gt "$CRITICAL" ]; then
        asterisk -rx "core restart when convenient"
    fi
fi
Enter fullscreen mode Exit fullscreen mode

Add to crontab:

*/5 * * * * /usr/local/bin/check_asterisk_channels.sh
Enter fullscreen mode Exit fullscreen mode

This runs every 5 minutes.

Section 4: Handling Real-World Scenarios

Scenario 1: Carrier Rate Limiting

Your carrier limits you to 50 channels but you're dialing 80 agents.

Solution: In sip-vicidial.conf:

[carrier1]
type = peer
host = 203.0.113.45
port = 5060
maxchannels = 50
Enter fullscreen mode Exit fullscreen mode

Then in ViciDial campaign settings, set Max Simultaneous Calls = 50. Excess calls queue automatically.

Monitor:

asterisk -rx "sip show peer carrier1" | grep -E "Maxchannels|Current"
Enter fullscreen mode Exit fullscreen mode

Scenario 2: Multiple Carriers with Different Limits 

[carrier1]
type = peer
host = 203.0.113.45
port = 5060
maxchannels = 100

[carrier2]
type = peer
host = 203.0.113.46
port = 5060
maxchannels = 50

[carrier3]
type = peer
host = 203.0.113.47
port = 5060
maxchannels = 75
Enter fullscreen mode Exit fullscreen mode

Total capacity: 225 channels. Distribute campaigns across carriers:

UPDATE vicidial_campaign 
SET carrier_id = 'carrier1' 
WHERE campaign_id = 'CAMP01';

UPDATE vicidial_campaign 
SET carrier_id = 'carrier2' 
WHERE campaign_id = 'CAMP02';
Enter fullscreen mode Exit fullscreen mode

Monitor aggregate usage:

asterisk -rx "sip show channels" | tail -1
Enter fullscreen mode Exit fullscreen mode

Scenario 3: Inbound Floods

During high inbound call volume, channels fill quickly.

Solution: Create a priority queue in extensions-vicidial.conf:

[from-carrier]
exten => 8005551234,1,Answer()
exten => 8005551234,2,Set(QUEUE_NAME=inbound_queue)
exten => 8005551234,3,Queue(${QUEUE_NAME},t)
exten => 8005551234,4,Hangup()
Enter fullscreen mode Exit fullscreen mode

With queues configured in queues.conf:

[inbound_queue]
music = default
strategy = rrmemory
timeout = 15
announce = queue-thankyou
member => SIP/2001
member => SIP/2002
member => SIP/2003
member => SIP/2004
Enter fullscreen mode Exit fullscreen mode

This prevents all channels from being consumed by unrouted calls.

Scenario 4: Resource Exhaustion from Memory Leaks

Over weeks, Asterisk consumes more memory and becomes slow.

Monitoring:

# Create a daily memory report
date >> /var/log/asterisk/memory_usage.log
ps aux | grep asterisk | grep -v grep >> /var/log/asterisk/memory_usage.log
asterisk -rx "memory show summary" >> /var/log/asterisk/memory_usage.log
Enter fullscreen mode Exit fullscreen mode

Solution: Implement a scheduled restart:

#!/bin/bash
# /usr/local/bin/restart_asterisk_weekly.sh

# Run every Sunday at 2 AM
0 2 * * 0 /usr/local/bin/restart_asterisk_weekly.sh

# Graceful restart
asterisk -rx "core restart when convenient"
sleep 300
systemctl restart asterisk
Enter fullscreen mode Exit fullscreen mode

Section 5: Optimization Techniques

Increase System File Descriptor Limits

Edit /etc/security/limits.conf:

asterisk soft nofile 65536
asterisk hard nofile 65536
asterisk soft nproc 65536
asterisk hard nproc 65536
Enter fullscreen mode Exit fullscreen mode

Verify after reboot:

cat /proc/$(pidof asterisk)/limits | grep "open files"
Enter fullscreen mode Exit fullscreen mode

Expected output:

Max open files            65536                65536                files
Enter fullscreen mode Exit fullscreen mode

Optimize RTP Port Range

In sip-vicidial.conf:

[general]
rtpstart = 10000
rtpend = 60000
Enter fullscreen mode Exit fullscreen mode

This allows up to 2,500 simultaneous calls (2 RTP ports per call). For 500+ agents, expand to:

rtpstart = 8000
rtpend = 65000
Enter fullscreen mode Exit fullscreen mode

Disable Unnecessary SIP Features

Remove overhead in sip-vicidial.conf:

[general]
srvlookup = no           ; Disable SRV lookups
alwaysauthreject = no    ; Reduce auth failures
disallow = all
allow = ulaw,alaw        ; Only allow codecs you use
videosupport = no        ; Disable video
Enter fullscreen mode Exit fullscreen mode

Use Outbound Proxies

In sip-vicidial.conf, route all traffic through a proxy:

[carrier1]
type = peer
host = 203.0.113.45
port = 5060
outboundproxy = 203.0.113.100:5060
Enter fullscreen mode Exit fullscreen mode

This centralizes call routing and prevents Asterisk from managing complex peer connections.

Implement Call State Caching

ViciDial's quick_closer.pl script caches call states. Optimize:

# /usr/share/astguiclient/quick_closer.pl interval (default 3 seconds)
# Lower value = more database queries but faster state updates
# Higher value = fewer queries but slightly stale data

# In the script around line 50:
$check_interval = 1;  # Check every 1 second (default is 3)
Enter fullscreen mode Exit fullscreen mode

Section 6: Troubleshooting

Issue: "All Circuits Busy" When Channels Show Available

Check for:

  1. Dead/zombie SIP channels not terminating properly 
asterisk -rx "sip show channels verbose"
Enter fullscreen mode Exit fullscreen mode

Look for channels showing 0 bytes sent/received for > 60 seconds.

Fix:

asterisk -rx "sip reload"
Enter fullscreen mode Exit fullscreen mode

Or restart specific peers:

asterisk -rx "sip notify myphone"
Enter fullscreen mode Exit fullscreen mode
  1. Peer not accepting new calls despite low count

Check peer status:

asterisk -rx "sip show peer carrier1"
Enter fullscreen mode Exit fullscreen mode

Look for Status: UNREACHABLE. Restart the peer:

asterisk -rx "sip qualify peer carrier1"
Enter fullscreen mode Exit fullscreen mode
  1. Channel limit mismatch between config and loaded settings

Reload SIP config:

asterisk -rx "sip reload"
Enter fullscreen mode Exit fullscreen mode

Verify the reload succeeded:

asterisk -rx "module show like chan_sip"
Enter fullscreen mode Exit fullscreen mode

Issue: Channels Exhausted Only During Peak Hours

Cause: Calls not releasing quickly enough.

Debug: Add call logging to extensions-vicidial.conf:

[from-campaign]
exten => _X.,1,Answer()
exten => _X.,2,Log(WARNING, "Call started to ${EXTEN}")
exten => _X.,3,Dial(SIP/${EXTEN}@carrier1,30,Tt)
exten => _X.,4,Log(WARNING, "Call ended to ${EXTEN}, result: ${DIALSTATUS}")
exten => _X.,5,Hangup()
Enter fullscreen mode Exit fullscreen mode

Check /var/log/asterisk/messages for unusually long call durations:

grep "Call started\|Call ended" /var/log/asterisk/messages | tail -100
Enter fullscreen mode Exit fullscreen mode

Fix: Lower connection timeouts:

exten => _X.,3,Dial(SIP/${EXTEN}@carrier1,20,Tt)
Enter fullscreen mode Exit fullscreen mode

Reduced from 30 to 20 seconds.

Issue: Specific Carrier Hits Limit While Others Are Empty

Cause: Uneven load distribution.

Check carrier assignment:

SELECT carrier_id, COUNT(*) 
FROM vicidial_log 
WHERE end_epoch = 0 
GROUP BY carrier_id;
Enter fullscreen mode Exit fullscreen mode

Solution: Adjust campaign carrier selection in ViciDial admin panel or load-balance via Asterisk:

[from-campaign]
exten => _X.,1,Set(CARRIER=$[${RANDOM} % 3 + 1])
exten => _X.,2,Dial(SIP/${EXTEN}@carrier${CARRIER},30,Tt)
exten => _X.,3,Hangup()
Enter fullscreen mode Exit fullscreen mode

This selects between carrier1, carrier2, carrier3 randomly.

Issue: Asterisk Process Uses Excessive Memory

Monitor memory over time:

watch -n 1 'ps aux | grep asterisk | grep -v grep | awk "{print \$6}"'
Enter fullscreen mode Exit fullscreen mode

Common causes:

  1. CDR database queue overflow — Check CDR settings: 
; In /etc/asterisk/cdr.conf
[general]
enable = yes
backend = odbc
Enter fullscreen mode Exit fullscreen mode

If ODBC connection fails, CDRs queue in memory. Fix:

mysql -u root -p asterisk << EOF
DELETE FROM cdr WHERE answer = 0 AND start > DATE_SUB(NOW(), INTERVAL 7 DAY);
EOF
Enter fullscreen mode Exit fullscreen mode
  1. Parking lot overflow — Check parked calls: 
asterisk -rx "parked calls"
Enter fullscreen mode Exit fullscreen mode

Clear old parked calls:

asterisk -rx "parkinglot flush"
Enter fullscreen mode Exit fullscreen mode
  1. RTP stream leaks — Restart: 
systemctl restart asterisk
Enter fullscreen mode Exit fullscreen mode

Issue: SIP Errors in Logs Despite Normal Channel Count

Example log:

[2024-01-15 14:32:15] WARNING[12345]: chan_sip.c:3245 sip_send_invite: Max retries exceeded on transmission of INVITE
[2024-01-15 14:32:16] WARNING[12345]: chan_sip.c:2156 handle_request_options: No supported header in OPTIONS
Enter fullscreen mode Exit fullscreen mode

Cause: Network congestion or carrier rejections.

Check carrier logs:

asterisk -rx "sip debug peer carrier1" | head -50
Enter fullscreen mode Exit fullscreen mode

Solution: Enable QoS and prioritize SIP traffic:

# On Linux, add to /etc/rc.local
tc qdisc add dev eth0 root handle 1: prio bands 3 priomap 0 0 0 1 2 2 2 2 2 2 2 2 2 2 2 2
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip dport 5060 0xffff flowid 1:1
tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip sport 5060 0xffff flowid 1:1
Enter fullscreen mode Exit fullscreen mode

Summary

The "All Circuits Busy" error is a symptom of channel exhaustion, not a root cause. Fixing it requires a multi-faceted approach:

  1. Diagnose accurately — Use sip show channels, database queries, and resource monitoring to identify bottlenecks
  2. Configure properly — Set appropriate maxchannels limits per peer and implement call gating
  3. Prevent overload — Use queues, session timers, and automatic restart mechanisms
  4. Monitor continuously — Check channel usage, memory, and SIP peer status
  5. Scale strategically — Add carriers, increase RTP port ranges, and optimize file descriptor limits

In production ViciDial systems, most "all circuits busy" errors occur due to:

  • Underestimated carrier channel limits (60% of cases)
  • Dead/zombie channels not releasing (25% of cases)
  • System resource exhaustion (10% of cases)
  • Configuration mismatches (5% of cases)

Implement the diagnostic steps from Section 1 first, then apply appropriate fixes from Sections 2-5. Test changes in a staging environment before production deployment. Most issues resolve with proper maxchannels configuration and call gating without requiring hardware upgrades.

Monitor proactively using the scripts provided. An alert at 80% channel capacity allows time to investigate before users experience dropped calls. Automated graceful restarts at 95% capacity prevent hard failures during traffic spikes.

Top comments (0)