ViciDial Troubleshooting Mega Guide

The top 20 most-discussed ViciDial problems — diagnosed and solved.

Covers ViciDial 2.14+ on ViciBox 11/12 (openSUSE Leap 15.x) and scratch installs on AlmaLinux 8/9.
Asterisk versions: 11, 13, 16, 18, 20. MariaDB 10.5–10.11.

Reading time: ~75 minutes

---

Table of Contents

1. Agent Going LAGGED / Auto-Paused
2. "0 Leads in the Dial Hopper"
3. One-Way Audio
4. Database Performance / Slow Reports
5. Missing Call Recordings
6. SIP Trunk Registration Failure
7. Disk Space Exhaustion
8. MySQL/MariaDB Table Crashes
9. Predictive Dialer Not Sending Calls
10. WebRTC / ViciPhone Not Working
11. Chrome JS Throttling Deep Dive
12. After-Hours / No-Agent Call Routing
13. Transfer / Conference Problems
14. Callback Scheduling Issues
15. Time Synchronization Errors
16. AMD (Answering Machine Detection) Tuning
17. Cron Jobs / Keepalive Dying
18. DAHDI / Timing Source Problems
19. Agent Login Failures
20. Recording Playback / Archive Issues

---

1. Agent Going LAGGED / Auto-Paused

Symptoms

• Agent screen shows status LAGGED after being in a background browser tab for several minutes.
• Agent is automatically paused and removed from call routing.
• The agent screen stops refreshing (timer freezes), then jumps forward when the tab regains focus.
• Managers see agents cycling between READY and PAUSED without any manual action.

Root Cause

Google Chrome throttles JavaScript timers in background tabs. This has evolved through several Chrome versions:

Chrome Version	Change	Impact on ViciDial
v57 (March 2017)	Background tabs limited to 1 timer wake-up per second	Minor — ViciDial's 1-second refresh still worked
v78 (October 2019)	Chained timers (5+ chains) throttled more aggressively	Occasional lag after extended background periods
v88 (January 2021)	Intensive Wake-Up Throttling — background tabs limited to 1 wake-up per minute after 5 minutes hidden	Breaks ViciDial completely — agent screen stops updating, session times out, agent goes LAGGED

The intensive throttling kicks in when ALL of these conditions are true:

• The page has been hidden (background tab) for more than 5 minutes
• The timer chain count is 5 or greater
• The page has been silent (no audio playing) for at least 30 seconds
• WebRTC is not in use

Diagnosis Steps

1. Confirm the problem is Chrome throttling (not network issues):

   # On the ViciDial server, check if the agent's session is timing out
   mysql -u cron -p1234 asterisk -e \
     "SELECT user, status, last_update_time FROM vicidial_live_agents WHERE user='AGENT_ID';"

If last_update_time is more than 30 seconds behind current time, the agent screen is not sending updates.

1. Check ViciDial version:

   grep 'build' /usr/share/astguiclient/ADMIN_keepalive_ALL.pl | head -3
   # Or check Admin → System Settings → Active Voicemail Server (shows SVN revision)

You need SVN revision 3390+ for the Agent Hidden Browser Sound fix, and SVN 3407+ for the EventSource Agent Screen Timer.

1. Check current System Settings:
   • Admin → System Settings → scroll to Agent Screen Timer section
   • Look for: Agent Screen Timer, Agent Hidden Browser Sound, Agent Hidden Browser Sound Seconds

Solution

There are five approaches, listed from most effective to least:

Option A: Agent Hidden Browser Sound (Recommended — SVN 3390+)

This plays an inaudible (or near-inaudible) sound file every N seconds when the agent tab is hidden, which prevents Chrome from classifying the tab as "silent" and triggering intensive throttling.

1. Go to Admin → System Settings
2. Set Agent Hidden Browser Sound to Y
3. Set Agent Hidden Browser Sound Seconds to 20
4. Save and have agents reload their browser

This works because Chrome exempts tabs that are playing audio from intensive throttling.

Option B: EventSource Agent Screen Timer (SVN 3407+)

Instead of using JavaScript setTimeout/setInterval (which Chrome throttles), this uses Server-Sent Events (EventSource), which are not subject to timer throttling.

1. Go to Admin → System Settings
2. Set Agent Screen Timer to EventSource
3. Save

The EventSource method opens a persistent HTTP connection from the browser to /agc/vdc_script_eventsource.php on the server. The server pushes timer events at the configured interval, bypassing Chrome's timer throttling entirely.

Option C: Use WebRTC/ViciPhone

When a WebRTC connection is active on the page, Chrome disables intensive throttling for that tab entirely. If you are already using ViciPhone (WebRTC softphone), agents will not experience LAGGED issues.

Option D: Chrome Policy (Enterprise/GPO)

For Windows environments with Group Policy management:

# Windows Registry (deploy via GPO):
# HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Chrome
# Add DWORD: IntensiveWakeUpThrottlingEnabled = 0

# Or via Chrome Enterprise policy JSON (Linux/Mac):
# /etc/opt/chrome/policies/managed/vicidial.json
{
  "IntensiveWakeUpThrottlingEnabled": false
}

Warning: Chrome 90+ removed the chrome://flags option for this. The enterprise policy still works but requires admin deployment. Chrome may deprecate this policy in future versions.

Option E: Switch to Firefox

Firefox does not implement the same aggressive intensive throttling. Background tabs are throttled to 1-second intervals (same as Chrome v57 behavior), which does not break ViciDial's refresh cycle.

Prevention

• Always deploy Option A or B on new ViciDial installations.
• Include the Chrome policy in your standard workstation image.
• Train agents: "Do not minimize or switch away from the ViciDial tab during your shift."
• Monitor for LAGGED events: query vicidial_agent_log for sub_status = 'LAGGED'.

Verify It Worked

-- Check for LAGGED events in the last 24 hours
SELECT user, event_time, pause_type
FROM vicidial_agent_log
WHERE event_time > NOW() - INTERVAL 24 HOUR
  AND sub_status = 'LAGGED'
ORDER BY event_time DESC;

If the count drops to zero after deploying the fix, it is working.

---

2. "0 Leads in the Dial Hopper"

Symptoms

• Campaign status page shows: "This campaign has 0 leads in the dial hopper"
• Agents sit in READY/WAITING but no calls go out.
• The hopper count stays at zero even though lists have thousands of leads.

Root Cause

Three primary causes, in order of frequency:

1. Local Call Time window — It is outside the callable hours for the leads' timezone. ViciDial checks the lead's timezone (based on area code/postal code), not the server's timezone.
2. No dialable statuses configured — The campaign only allows certain statuses (e.g., NEW) but all leads have already been called and are in a different status (e.g., A, NA, B).
3. All leads already called — Every lead with an allowed dial status has called_since_last_reset = Y.
4. Lead filter blocking all leads — A misconfigured lead filter can prevent all leads from loading. A bad filter on ONE campaign can affect ALL campaigns on the server.

Diagnosis Steps

1. Check the hopper directly:

   # Open in browser:
   http://YOUR_SERVER/vicidial/AST_VDhopper.php

This shows the current hopper contents and the last hopper load attempt with diagnostics.

1. Check Local Call Time:

   -- What call time is assigned to the campaign?
   SELECT campaign_id, local_call_time FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

   -- What does that call time allow?
   SELECT * FROM vicidial_call_times
   WHERE call_time_id = 'THE_CALL_TIME_ID';

For testing, temporarily set the campaign's Local Call Time to 24hours (allows calling any time).

1. Check dialable statuses:

   -- What statuses is this campaign allowed to dial?
   SELECT campaign_id, dial_statuses FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

   -- What statuses do the leads actually have?
   SELECT status, COUNT(*) AS cnt
   FROM vicidial_list
   WHERE list_id IN (
     SELECT list_id FROM vicidial_lists
     WHERE campaign_id = 'YOUR_CAMPAIGN' AND active = 'Y'
   )
   GROUP BY status ORDER BY cnt DESC;

If all leads are in status A (Answering Machine) but your dial statuses only include NEW, no leads will load.

1. Check lead filters:

   SELECT campaign_id, lead_filter_id FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

Set lead_filter_id to NONE temporarily to test. A malformed SQL filter can silently block all leads.

1. Check list active status:

   SELECT list_id, list_name, active, list_lastcalldate
   FROM vicidial_lists
   WHERE campaign_id = 'YOUR_CAMPAIGN';

Lists must have active = 'Y'.

1. Check the hopper loading script is running:

   ps aux | grep AST_VDhopper

If AST_VDhopper.pl is not running, leads will never load into the hopper.

Solution

For Call Time issues:

• Admin → Campaigns → [Your Campaign] → Detail View → Local Call Time → set to 24hours for testing
• For production, create a call time that matches your actual operating hours and the timezone of your leads

For dialable status issues:

• Admin → Campaigns → [Your Campaign] → Detail View → Dial Statuses
• Check the boxes for all statuses you want to re-dial (e.g., NA, B, A, NEW, CALLBK)

For "all leads called" issues:

• Admin → Lists → [Your List] → Reset List (resets called_since_last_reset to N for all leads)
• Or from MySQL:

  UPDATE vicidial_list SET called_since_last_reset = 'N'
  WHERE list_id = 'YOUR_LIST_ID' AND status IN ('NA','B');

For lead filter issues:

• Set the campaign's lead filter to NONE
• If you need a filter, test the SQL manually first:

  -- Whatever your filter SQL is, test it:
  SELECT COUNT(*) FROM vicidial_list WHERE list_id = 'YOUR_LIST' AND (YOUR_FILTER_SQL);

Prevention

• Always verify Local Call Time settings when importing leads from a new timezone.
• After uploading a list, check the hopper page within 2 minutes to confirm leads are loading.
• Use the AST_VDhopper.php page as part of your daily campaign startup checklist.

Verify It Worked

# Watch the hopper fill in real-time (runs every minute via cron):
watch -n 10 "mysql -u cron -p1234 asterisk -e \
  \"SELECT campaign_id, COUNT(*) FROM vicidial_hopper GROUP BY campaign_id;\""

---

3. One-Way Audio

Symptoms

• The agent can hear the customer but the customer cannot hear the agent (or vice versa).
• Audio works on some calls but not others.
• Audio works for the first few seconds then drops.
• Internal calls between extensions work fine, but external SIP trunk calls have one-way audio.

Root Cause

One-way audio is almost always caused by NAT/firewall issues with RTP (media) packets. SIP handles call signaling on port 5060, but the actual voice audio travels over RTP on dynamic UDP ports (typically 10000–20000). If a firewall or NAT device blocks or fails to route these RTP packets, one direction of audio is lost.

Common specific causes:

1. Missing externip/localnet settings in sip.conf when Asterisk is behind NAT
2. SIP ALG (Application Layer Gateway) on the router rewriting SIP headers incorrectly
3. RTP port range not forwarded through the firewall
4. canreinvite=yes (or directmedia=yes in PJSIP) allowing Asterisk to redirect RTP directly between endpoints that cannot reach each other

Diagnosis Steps

Step 1: Identify the direction

Scenario	Likely Cause
Agent hears customer, customer hears nothing	RTP from Asterisk to carrier/customer is blocked
Customer hears agent, agent hears nothing	RTP from carrier to Asterisk is blocked
Both directions silent	SIP signaling worked, but no RTP at all — check codec mismatch or complete firewall block

Step 2: Capture SIP/RTP traffic with sngrep

# Install sngrep if not present
# ViciBox/openSUSE:
zypper install sngrep
# AlmaLinux:
dnf install sngrep

# Capture SIP + RTP traffic:
sngrep -r

# Filter to a specific call (press F7 for filter dialog)
# Or filter by IP on the command line:
sngrep -r -d eth0 host CARRIER_IP

In sngrep:

• Press Enter on an INVITE to see the full SIP dialog
• Press F2 to see the SDP (codec and IP/port negotiation)
• Press F3 to see RTP stream statistics
• Press F2 again to save a .pcap file for Wireshark analysis

Step 3: Check SDP for wrong IP

Look at the SDP body in the INVITE and 200 OK:

o=- 12345 12345 IN IP4 10.0.0.5    <-- PROBLEM: private IP in SDP
c=IN IP4 10.0.0.5                    <-- PROBLEM: private IP in SDP
m=audio 18450 RTP/AVP 0 8 101

If you see a private IP (10.x, 172.16-31.x, 192.168.x) in the SDP when the server is behind NAT, Asterisk is advertising its private IP instead of its public IP. The remote side tries to send RTP to the private IP and fails.

Step 4: tcpdump for RTP verification

# Capture RTP traffic on the port range
tcpdump -i eth0 udp portrange 10000-20000 -c 100 -nn

# If you see packets going OUT but none coming IN (or vice versa), it is a NAT/firewall issue

Solution

Fix 1: Configure externip and localnet in sip.conf

Edit /etc/asterisk/sip.conf:

[general]
; Your public IP (or FQDN if dynamic IP)
externip=203.0.113.50
; Or for dynamic IP:
; externhost=mydialer.example.com
; externrefresh=60

; Your local network(s) — Asterisk will use the internal IP for
; calls within these networks, and externip for everything else
localnet=10.0.0.0/8
localnet=172.16.0.0/12
localnet=192.168.0.0/16

; NAT settings
nat=force_rport,comedia

; Prevent Asterisk from redirecting RTP between endpoints
canreinvite=no
; (In Asterisk 12+, this is called directmedia=no)
directmedia=no

After editing:

asterisk -rx "sip reload"

For PJSIP (Asterisk 16+/ViciBox 11+):

Edit /etc/asterisk/pjsip.conf:

[transport-udp]
type=transport
protocol=udp
bind=0.0.0.0
external_media_address=203.0.113.50
external_signaling_address=203.0.113.50
local_net=10.0.0.0/8
local_net=172.16.0.0/12
local_net=192.168.0.0/16

After editing:

asterisk -rx "pjsip reload res_pjsip.so"

Fix 2: Open RTP port range in firewall

# Check current rtp.conf settings:
cat /etc/asterisk/rtp.conf
# [general]
# rtpstart=10000
# rtpend=20000

# Open the range in firewall (firewalld on AlmaLinux):
firewall-cmd --permanent --add-port=10000-20000/udp
firewall-cmd --reload

# Or iptables (ViciBox/openSUSE):
iptables -A INPUT -p udp --dport 10000:20000 -j ACCEPT

Fix 3: Disable SIP ALG on the router

SIP ALG rewrites SIP headers and often breaks NAT traversal. Disable it on your router/firewall:

• Most consumer routers: Look for "SIP ALG" or "SIP Passthrough" in the NAT/firewall settings and disable it
• pfSense: System → Advanced → Firewall & NAT → uncheck "Disable Firewall Scrub"
• MikroTik: /ip firewall service-port disable sip

Fix 4: Add nat settings to individual SIP peers

If the global setting is not enough, add NAT settings to the specific carrier peer in /etc/asterisk/sip-vicidial.conf:

[my_carrier]
type=peer
host=sip.carrier.com
nat=force_rport,comedia
canreinvite=no
qualify=yes
disallow=all
allow=ulaw
allow=alaw

Prevention

• Always configure externip/localnet during initial setup if the server is behind NAT
• Test bidirectional audio immediately after adding a new SIP trunk
• Use sngrep -r to verify RTP flow on the first test call
• Document your firewall rules — RTP port range must always be open

Verify It Worked

# Make a test call, then check RTP stats in sngrep:
sngrep -r
# Select the call → F3 → verify packets are flowing in both directions
# Both "Src packets" and "Dst packets" should be non-zero

---

4. Database Performance / Slow Reports

Symptoms

• Admin reports take 30+ seconds or time out entirely.
• Agent screens lag or show delayed updates.
• The server load average is high (10+) and mysqld is consuming most CPU.
• SHOW PROCESSLIST shows many long-running queries.

Root Cause

ViciDial is extremely database-intensive. Every agent screen refresh queries multiple tables. With 50+ agents, the database handles thousands of queries per second. Common causes of slowness:

1. Default my.cnf settings — ViciBox ships with conservative MySQL settings that do not scale
2. Missing indexes — Key queries do full table scans on large tables
3. Unarchived log tables — vicidial_log, vicidial_closer_log, and call_log grow indefinitely
4. MyISAM table locks — ViciDial still uses MyISAM for most tables, which uses table-level locking
5. Slow storage — Spinning disks cannot keep up with ViciDial's I/O patterns

Diagnosis Steps

1. Check current database size:

   SELECT table_schema,
          ROUND(SUM(data_length + index_length) / 1024 / 1024, 1) AS size_mb,
          SUM(table_rows) AS total_rows
   FROM information_schema.tables
   WHERE table_schema = 'asterisk'
   GROUP BY table_schema;

1. Find the largest tables:

   SELECT table_name,
          ROUND((data_length + index_length) / 1024 / 1024, 1) AS size_mb,
          table_rows
   FROM information_schema.tables
   WHERE table_schema = 'asterisk'
   ORDER BY data_length + index_length DESC
   LIMIT 20;

If vicidial_log, vicidial_closer_log, call_log, or recording_log are hundreds of MB or multi-GB, they need archiving.

1. Check for long-running queries:

   SHOW FULL PROCESSLIST;
   -- Or more specifically:
   SELECT id, user, host, db, command, time, state, LEFT(info, 200) AS query
   FROM information_schema.processlist
   WHERE command != 'Sleep' AND time > 5
   ORDER BY time DESC;

1. Enable the slow query log temporarily:

   SET GLOBAL slow_query_log = 1;
   SET GLOBAL long_query_time = 2;
   SET GLOBAL slow_query_log_file = '/var/log/mysql/slow.log';

Review after an hour:

   mysqldumpslow -s t /var/log/mysql/slow.log | head -40

1. Check my.cnf values:

   SHOW VARIABLES LIKE 'key_buffer_size';
   SHOW VARIABLES LIKE 'innodb_buffer_pool_size';
   SHOW VARIABLES LIKE 'table_open_cache';
   SHOW VARIABLES LIKE 'max_connections';
   SHOW VARIABLES LIKE 'query_cache_size';
   SHOW VARIABLES LIKE 'query_cache_type';

Solution

Fix 1: Tune my.cnf

Edit /etc/my.cnf or /etc/my.cnf.d/server.cnf:

[mysqld]
# === Memory Settings ===
# key_buffer_size: up to 1/4 of total RAM for MyISAM indexes
# For a 16GB server:
key_buffer_size = 4G

# innodb_buffer_pool_size: 50-70% of RAM if using InnoDB tables
# ViciDial uses mostly MyISAM, so keep this moderate
innodb_buffer_pool_size = 2G

# query_cache: helps with repeated identical queries
# Note: deprecated in MySQL 8.0, still works in MariaDB 10.x
query_cache_size = 256M
query_cache_type = 1
query_cache_limit = 4M

# === Connection Settings ===
max_connections = 2000
table_open_cache = 8192
thread_cache_size = 128
tmp_table_size = 256M
max_heap_table_size = 256M

# === Logging ===
# Kill long-running queries from misbehaving report scripts
max_statement_time = 300

# === MyISAM Settings ===
myisam_sort_buffer_size = 128M
myisam_repair_threads = 1

Restart MariaDB:

systemctl restart mariadb
# or
systemctl restart mysql

Fix 2: Add missing indexes

These indexes dramatically speed up the most common ViciDial queries:

-- Speed up agent log lookups (used by many admin reports)
CREATE INDEX idx_uniqueid ON vicidial_agent_log (uniqueid);

-- Speed up call_log queries by channel group and time
CREATE INDEX idx_changrp_starttime ON call_log (channel_group, start_time);

-- Speed up closer_log queries by campaign and date
CREATE INDEX idx_campaign_calldate_status
  ON vicidial_closer_log (campaign_id, call_date, status);

-- Speed up recording lookups
CREATE INDEX idx_reclog_start ON recording_log (start_time);

-- Speed up vicidial_log queries by date range
CREATE INDEX idx_vlog_calldate ON vicidial_log (call_date);

-- Speed up lead lookups by phone number
CREATE INDEX idx_vlist_phone ON vicidial_list (phone_number);

Fix 3: Archive old data

Use ViciDial's built-in archiving script:

# Archive data older than 90 days (adjust --days as needed)
/usr/share/astguiclient/ADMIN_archive_log_tables.pl \
  --days=90 --closer-log --vlog-daily --carrier-daily

# For daily archiving (recommended), add to cron:
# Archive call_log, vicidial_log_extended, vicidial_dial_log, vicidial_drop_log daily
# Keep only last 24 hours in these tables
3 1 * * * /usr/share/astguiclient/ADMIN_archive_log_tables.pl --daily --closer-log --carrier-daily --vlog-daily 2>&1

The script creates _archive copies of each table (e.g., vicidial_log_archive) and moves old records there. Reports still work on archived data via the "Archived" report options in the admin UI.

Fix 4: Optimize tables periodically

# Built-in ViciDial table optimizer — runs via cron at 3:01 AM by default
# Verify it is in crontab:
crontab -l | grep optimize

# If missing, add:
# 1 3 * * * /usr/share/astguiclient/AST_DB_optimize.pl 2>&1

You can also run it manually:

mysqlcheck -u cron -p1234 --optimize asterisk

Prevention

• Set up daily archiving in cron from day one
• Run the mysql-tuning.sh script after installation: /usr/share/astguiclient/extras/mysql-tuning.sh
• Use SSDs — ViciDial's random I/O pattern is devastating to spinning disks
• Monitor table sizes weekly

Verify It Worked

-- Check if archiving is working (archive tables should have data)
SELECT table_name, table_rows,
       ROUND((data_length + index_length) / 1024 / 1024, 1) AS size_mb
FROM information_schema.tables
WHERE table_schema = 'asterisk'
  AND table_name LIKE '%_archive'
ORDER BY table_rows DESC;

---

5. Missing Call Recordings

Symptoms

• Calls show in reports but clicking the recording link gives "Not Found" or a blank player.
• Recordings exist for some calls but not others.
• The /var/spool/asterisk/monitorDONE/ directory is empty or contains only .wav files (no MP3s).
• Recording links point to .mp3 files that do not exist on disk.

Root Cause

ViciDial's recording pipeline has multiple stages, and a failure at any stage causes missing recordings:

Asterisk records call (MixMonitor/Monitor)
  → /var/spool/asterisk/monitor/  (raw .wav files, dual-channel)
  → AST_CRON_audio_1_move_mix.pl  (mixes channels, moves to monitorDONE/ORIG/)
  → AST_CRON_audio_2_compress.pl  (converts .wav → .mp3, places in monitorDONE/MP3/)
  → AST_CRON_audio_3_ftp.pl       (uploads to FTP archive server, optional)
  → Recording link in DB updated to point to the final file location

Common failure points:

1. Cron scripts not running — AST_CRON_audio_1_move_mix.pl or AST_CRON_audio_2_compress.pl not in crontab
2. Missing --MP3 flag — AST_CRON_audio_2_compress.pl must be called with --MP3 to produce MP3 files
3. Disk full — No space to write converted files
4. Missing SOX or LAME — The audio conversion tools are not installed
5. Permission issues — The cron user cannot write to monitorDONE directories
6. Campaign recording setting disabled — The campaign has recording turned off

Diagnosis Steps

1. Check if recordings exist on disk:

   # Check raw recordings (pre-conversion)
   ls -la /var/spool/asterisk/monitor/ | tail -10

   # Check mixed originals
   ls -la /var/spool/asterisk/monitorDONE/ORIG/ | tail -10

   # Check compressed MP3s
   ls -la /var/spool/asterisk/monitorDONE/MP3/ | tail -10

   # Check FTP staging
   ls -la /var/spool/asterisk/monitorDONE/FTP/ | tail -10

1. Check if cron scripts are running:

   crontab -l | grep -i audio
   # You should see entries like:
   # */3 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl
   # */3 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3
   # */3 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl

1. Run the scripts manually with debug output:

   /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl --debugX
   /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3 --debugX

Look for errors about missing tools, permissions, or disk space.

1. Check for required audio tools:

   which sox && sox --version
   which lame && lame --version
   # On ViciBox/openSUSE:
   which ffmpeg && ffmpeg -version

1. Check campaign recording setting:

   SELECT campaign_id, campaign_rec, campaign_rec_filename
   FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

campaign_rec should be ONDEMAND or ALLCALLS or ALLFORCE.

1. Check the recording_log table:

   SELECT recording_id, channel, server_ip, filename, location, length_in_sec
   FROM recording_log
   WHERE start_time > NOW() - INTERVAL 1 HOUR
   ORDER BY start_time DESC LIMIT 10;

If location is empty or points to a non-existent file, the pipeline is broken.

1. Check disk space:

   df -h /var/spool/asterisk/

Solution

Fix 1: Add missing cron entries

crontab -e
# Add these entries (adjust timing offsets to avoid overlap):

### recording mixing/compressing/ftping scripts
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl
1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3
2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3

Fix 2: Install missing audio tools

# ViciBox/openSUSE:
zypper install sox lame

# AlmaLinux 8/9:
dnf install sox
dnf install --enablerepo=epel lame

# Verify:
sox --version
lame --version

Fix 3: Fix permissions

chown -R asterisk:asterisk /var/spool/asterisk/monitor/
chown -R asterisk:asterisk /var/spool/asterisk/monitorDONE/
chmod -R 755 /var/spool/asterisk/monitorDONE/

Fix 4: Process backlogged recordings

If you have a backlog of unprocessed .wav files:

# Run the scripts sequentially to catch up
/usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl --debugX
sleep 5
/usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3 --debugX

Prevention

• Verify recording cron entries after every ViciDial upgrade
• Monitor disk space on /var/spool/asterisk/ with a cron alert
• Clean up old ORIG .wav files (they are the largest):

  # Delete ORIG wav files older than 7 days
  find /var/spool/asterisk/monitorDONE/ORIG -type f -name "*.wav" -mtime +7 -delete

Verify It Worked

# Make a test call, wait 5 minutes, then check:
ls -lt /var/spool/asterisk/monitorDONE/MP3/ | head -5
# You should see a fresh .mp3 file from the last few minutes

---

6. SIP Trunk Registration Failure

Symptoms

• sip show peers in Asterisk CLI shows the trunk as UNREACHABLE or Unregistered.
• Outbound calls fail with "All circuits are busy" or "Congestion".
• The Asterisk log shows Registration ... failed or No matching peer found.
• The carrier reports no registration attempts from your IP.

Root Cause

SIP trunk registration can fail due to:

1. Wrong credentials (username/password/host)
2. Incorrect registration string syntax
3. IP authentication vs. digest authentication mismatch
4. Firewall blocking port 5060
5. DNS resolution failure
6. Codec mismatch in the SIP negotiation

Diagnosis Steps

1. Check current registration status:

   asterisk -rx "sip show registry"
   # Shows all registration attempts and their status

   asterisk -rx "sip show peers"
   # Shows all SIP peers and their status (OK/UNREACHABLE/Unregistered)

1. Enable SIP debug temporarily:

   asterisk -rx "sip set debug on"
   # Watch the Asterisk console for registration REGISTER/401/200 exchanges
   # Remember to turn it off when done:
   asterisk -rx "sip set debug off"

1. Check DNS resolution:

   dig sip.carrier.com
   nslookup sip.carrier.com
   # Ensure the carrier's SIP server hostname resolves

1. Check firewall:

   # Test SIP port connectivity to the carrier
   nc -zuv sip.carrier.com 5060
   # Or with nmap:
   nmap -sU -p 5060 sip.carrier.com

1. Check the ViciDial admin carrier config:
   • Admin → Carriers → [Your Carrier]
   • Note the fields: Server IP, Protocol, Registration String, Account Entry

Solution

Understanding ViciDial Carrier Configuration

ViciDial writes carrier settings to /etc/asterisk/sip-vicidial.conf. The admin UI has these key fields:

Registration String — Used ONLY when the carrier requires username/password authentication (digest auth). Format:

register => USERNAME:PASSWORD@SIP_SERVER_IP:PORT

Example:

register => myaccount:s3cretP@[e-mail]:5060

If your carrier uses IP authentication (they whitelist your server's IP), leave the registration string blank. Adding a registration string when the carrier does not expect one causes errors.

Account Entry — The SIP peer definition. Example for digest auth:

[my_carrier]
type=peer
host=sip.carrier.com
port=5060
username=myaccount
secret=s3cretP@ss
fromuser=myaccount
fromdomain=sip.carrier.com
nat=force_rport,comedia
canreinvite=no
qualify=yes
disallow=all
allow=ulaw
allow=alaw
context=trunkinbound

Example for IP authentication (no username/password):

[my_carrier]
type=peer
host=sip.carrier.com
port=5060
nat=force_rport,comedia
canreinvite=no
qualify=yes
insecure=port,invite
disallow=all
allow=ulaw
allow=alaw
context=trunkinbound

Server IP — This is confusing in ViciDial. On a single-server install, set this to your ViciDial server's own IP address, NOT the carrier's IP.

After making changes:

# Reload SIP configuration:
asterisk -rx "sip reload"

# Check registration:
asterisk -rx "sip show registry"

# Check peer status:
asterisk -rx "sip show peers" | grep my_carrier

Common carrier-specific notes:

Telnyx: Uses IP authentication by default. Leave registration string blank. Set insecure=port,invite in the account entry.

Twilio: Uses Elastic SIP Trunking with IP authentication. Origination URI format for outbound. No registration needed.

VoIP.ms: Uses digest authentication. Registration string required. Format: register => SUBACCOUNT:PASSWORD@VOIPMS_SERVER:5060

Prevention

• Always test a new trunk with a single manual call before putting it into production
• Set qualify=yes so Asterisk actively monitors trunk health
• Monitor sip show peers output in your monitoring system

Verify It Worked

# Registration should show "Registered"
asterisk -rx "sip show registry"

# Peer should show "OK" with a latency value
asterisk -rx "sip show peers" | grep carrier_name

# Make a test call
asterisk -rx "originate SIP/my_carrier/15551234567 extension s@from-internal"

---

7. Disk Space Exhaustion

Symptoms

• ViciDial stops functioning — agent screens freeze, calls drop mid-conversation.
• MySQL crashes with "disk full" errors.
• df -h shows / or /var at 100%.
• New recordings are not saved.
• The system log shows write errors.

Root Cause

ViciDial generates large amounts of data:

• Call recordings: ~1 MB per minute of call time (MP3), ~10 MB per minute (WAV)
• Asterisk logs: /var/log/asterisk/messages can grow to multiple GB
• MySQL tables: sip_packets (if SIP capture is enabled) can grow 5+ GB per day
• monitorDONE/ORIG: Original WAV files accumulate if cleanup is not configured
• astguiclient logs: /var/log/astguiclient/ can fill with debug logs

Diagnosis Steps

# Overall disk usage
df -h

# Find the biggest directories
du -sh /var/spool/asterisk/* | sort -rh | head -10
du -sh /var/log/* | sort -rh | head -10

# Check recording directories specifically
du -sh /var/spool/asterisk/monitorDONE/
du -sh /var/spool/asterisk/monitorDONE/ORIG/
du -sh /var/spool/asterisk/monitorDONE/MP3/
du -sh /var/spool/asterisk/monitorDONE/FTP/

# Check MySQL data directory
du -sh /var/lib/mysql/asterisk/

# Check sip_packets table size (common offender)
mysql -u cron -p1234 asterisk -e \
  "SELECT table_name, ROUND((data_length + index_length)/1024/1024,1) AS size_mb \
   FROM information_schema.tables \
   WHERE table_schema='asterisk' AND table_name='sip_packets';"

# Check Asterisk log size
ls -lh /var/log/asterisk/messages*

Solution

Immediate relief (when disk is at 100%)

# 1. Truncate Asterisk log (safe to do while running)
> /var/log/asterisk/messages

# 2. Delete old ORIG wav files (keep MP3s, delete original WAVs older than 3 days)
find /var/spool/asterisk/monitorDONE/ORIG -type f -name "*.wav" -mtime +3 -delete

# 3. Truncate sip_packets if it is large (this data is ephemeral)
mysql -u cron -p1234 asterisk -e "TRUNCATE TABLE sip_packets;"

# 4. Clean old astguiclient logs
find /var/log/astguiclient/ -type f -mtime +2 -delete

# 5. Clean old Asterisk logs
find /var/log/asterisk/ -type f -mtime +7 -delete

# 6. Rotate Asterisk logs now
asterisk -rx "logger rotate"

Permanent recording cleanup cron

crontab -e
# Add these cleanup entries:

### Delete ORIG wav files older than 7 days
0 1 * * * find /var/spool/asterisk/monitorDONE/ORIG -type f -name "*.wav" -mtime +7 -delete

### Delete MP3 recordings older than 90 days (adjust retention as needed)
0 2 * * * find /var/spool/asterisk/monitorDONE/MP3 -type f -name "*.mp3" -mtime +90 -delete

### Clean old astguiclient logs older than 2 days
0 3 * * * find /var/log/astguiclient/ -type f -mtime +2 -delete

### Rotate and clean Asterisk logs older than 7 days
0 4 * * * find /var/log/asterisk/ -type f -mtime +7 -delete

Disable SIP capture (if enabled and not needed)

# Check if sip_capture_daemon is running
systemctl status sip_capture_daemon 2>/dev/null

# If running and not needed:
systemctl stop sip_capture_daemon
systemctl disable sip_capture_daemon

# Truncate the table
mysql -u cron -p1234 asterisk -e "TRUNCATE TABLE sip_packets;"

Asterisk log rotation (logrotate)

Create /etc/logrotate.d/asterisk:

/var/log/asterisk/messages {
    weekly
    rotate 4
    compress
    missingok
    notifempty
    postrotate
        /usr/sbin/asterisk -rx 'logger rotate' > /dev/null 2>&1
    endscript
}

Prevention

• Set up recording cleanup cron on day one of every new installation
• Archive recordings to a remote FTP/SFTP server using AST_CRON_audio_3_ftp.pl
• Monitor disk usage with a simple cron alert:

  # Alert when disk is over 85%
  0 * * * * [ $(df / --output=pcent | tail -1 | tr -dc '0-9') -gt 85 ] && echo "DISK WARNING: $(df -h /)" | mail -s "ViciDial Disk Alert" [e-mail]

• Disable sip_capture_daemon unless you are actively debugging SIP issues

Verify It Worked

# Check disk usage after cleanup
df -h /

# Verify cron jobs are in place
crontab -l | grep -E "(monitorDONE|astguiclient|asterisk.*delete)"

---

8. MySQL/MariaDB Table Crashes

Symptoms

• ViciDial shows errors like "Table 'vicidial_live_agents' is marked as crashed and should be repaired"
• Agent screens display database errors or fail to load.
• Reports return empty results or error messages.
• MySQL error log shows "Got error 127" or "Got error 134" or "Incorrect key file" messages.

Root Cause

ViciDial uses MyISAM as the default storage engine for most tables. MyISAM does not support transactions or crash recovery. Tables can become corrupted by:

1. Power loss / hard shutdown — If MySQL is writing to a MyISAM table during a power failure, the table is corrupted
2. Disk full — A write that fails mid-operation due to disk space leaves the table in an inconsistent state
3. Killed MySQL process — kill -9 on mysqld can corrupt open tables
4. Hardware failure — Bad RAM, failing disk
5. Concurrent access issues — Heavy writes during large SELECT queries can cause index corruption

Diagnosis Steps

1. Check MySQL error log:

   # ViciBox/openSUSE:
   tail -100 /var/log/mysql/mysqld.log
   # AlmaLinux:
   tail -100 /var/log/mariadb/mariadb.log

1. Check all tables for errors:

   # Check all tables in the asterisk database
   mysqlcheck -u cron -p1234 --check asterisk

1. Check a specific table:

   CHECK TABLE vicidial_live_agents;
   CHECK TABLE vicidial_log;
   CHECK TABLE call_log;

1. Identify which tables are MyISAM:

   SELECT table_name, engine
   FROM information_schema.tables
   WHERE table_schema = 'asterisk' AND engine = 'MyISAM'
   ORDER BY table_name;

Solution

Quick repair (while MySQL is running)

# Repair all tables in the asterisk database:
mysqlcheck -u cron -p1234 --auto-repair asterisk

# Or repair a specific table:
mysql -u cron -p1234 asterisk -e "REPAIR TABLE vicidial_live_agents;"
mysql -u cron -p1234 asterisk -e "REPAIR TABLE vicidial_log;"
mysql -u cron -p1234 asterisk -e "REPAIR TABLE call_log;"

If quick repair fails — use myisamchk

# Stop MySQL first
systemctl stop mariadb

# Navigate to the data directory
cd /var/lib/mysql/asterisk/

# Step 1: Quick recovery (try this first)
myisamchk -r -q vicidial_live_agents

# Step 2: Full recovery (if quick fails)
myisamchk -r vicidial_live_agents

# Step 3: Safe recovery (last resort — slow but handles edge cases)
myisamchk --safe-recover vicidial_live_agents

# Fix permissions after repair
chown mysql:mysql /var/lib/mysql/asterisk/*

# Start MySQL
systemctl start mariadb

Repair all tables at once

# While MySQL is running — repairs all databases
mysqlcheck -u cron -p1234 --auto-repair --all-databases

Prevention

1. Use a UPS — The single most effective prevention measure for MyISAM corruption
2. Always shut down MySQL properly: systemctl stop mariadb (never kill -9)
3. Monitor disk space — A full disk during writes guarantees corruption
4. Enable binary logging for point-in-time recovery:

   # In /etc/my.cnf:
   [mysqld]
   log-bin = mysql-bin
   expire_logs_days = 7

1. Schedule regular checks:

   # Add to cron — weekly check of all tables
   0 5 * * 0 mysqlcheck -u cron -p1234 --auto-repair asterisk >> /var/log/mysqlcheck.log 2>&1

Verify It Worked

# Check all tables — should show "OK" for every table
mysqlcheck -u cron -p1234 --check asterisk 2>&1 | grep -v OK
# If this produces no output, all tables are healthy

---

9. Predictive Dialer Not Sending Calls

Symptoms

• Agents are in READY/WAITING status but no calls are being placed.
• The hopper has leads (non-zero count) but nothing dials.
• The Real-Time report shows 0 calls ringing and the dialer appears frozen.
• vicidial_auto_calls table contains stale/stuck entries.

Root Cause

The predictive dialer depends on several Perl scripts running via cron, and on clean data in key database tables:

1. Missing cron jobs — AST_VDadapt.pl (adaptive algorithm) or AST_VDauto_dial.pl (actual dialing) not running
2. Campaign dial method/level misconfigured — Auto Dial Level set to 0, or dial method is MANUAL
3. Orphaned records in vicidial_auto_calls — Stale call records that make the dialer think lines are in use
4. Asterisk-Perl module version mismatch — Only asterisk-perl 0.08 works correctly with some ViciDial versions
5. ADMIN_keepalive_ALL.pl not running — The master process that keeps all dialer scripts alive

Diagnosis Steps

1. Check if dialer processes are running:

   ps aux | grep AST_VDauto_dial
   ps aux | grep AST_VDadapt
   ps aux | grep AST_VDhopper
   ps aux | grep ADMIN_keepalive

All four should show running processes.

1. Check campaign settings:

   SELECT campaign_id, dial_method, auto_dial_level, active,
          adaptive_maximum_level, adaptive_intensity
   FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

• dial_method should be RATIO, ADAPT_HARD_LIMIT, ADAPT_TAPERED, or ADAPT_AVERAGE (NOT MANUAL or INBOUND_MAN)
• auto_dial_level must be > 0 for ratio mode (typically 1.0 to 3.0)
• For adaptive methods, auto_dial_level is managed automatically but adaptive_maximum_level must be > 0

1. Check for stuck calls in vicidial_auto_calls:

   -- Find calls that have been "active" for an impossibly long time
   SELECT auto_call_id, server_ip, campaign_id, status, call_time,
          TIMESTAMPDIFF(MINUTE, call_time, NOW()) AS minutes_old
   FROM vicidial_auto_calls
   WHERE call_time < NOW() - INTERVAL 30 MINUTE;

If you see entries older than 30 minutes, they are orphaned and should be cleaned.

1. Check the hopper has leads:

   SELECT campaign_id, COUNT(*) as hopper_count
   FROM vicidial_hopper
   GROUP BY campaign_id;

1. Check the keepalive processes:

   # Run keepalive manually to see what it restarts
   /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

Solution

Fix 1: Clean orphaned vicidial_auto_calls

-- Delete stuck calls older than 30 minutes
DELETE FROM vicidial_auto_calls
WHERE call_time < NOW() - INTERVAL 30 MINUTE;

If this is a recurring problem, the uniqueid column may be too narrow. Fix it:

ALTER TABLE vicidial_auto_calls MODIFY uniqueid VARCHAR(40);
ALTER TABLE vicidial_log MODIFY uniqueid VARCHAR(40);
ALTER TABLE call_log MODIFY uniqueid VARCHAR(40);

Fix 2: Verify and fix cron entries

crontab -l | grep -E "(VDauto|VDadapt|VDhopper|keepalive)"

# Essential entries that MUST exist:
# * * * * * /usr/share/astguiclient/ADMIN_keepalive_ALL.pl
# * * * * * /usr/share/astguiclient/AST_VDhopper.pl -q

The keepalive script automatically starts AST_VDauto_dial.pl and AST_VDadapt.pl — they do not need their own cron entries. But ADMIN_keepalive_ALL.pl itself MUST be in cron.

Fix 3: Set correct campaign dial settings

For predictive dialing:

• Dial Method: RATIO (simplest) or ADAPT_HARD_LIMIT (recommended for compliance)
• Auto Dial Level: Start at 1.0 (1 line per agent), increase gradually
• For adaptive methods:
  • Adaptive Maximum Level: 3.0 (maximum lines per agent)
  • Adaptive Dropped Percentage: 3.0 (target max drop rate)
  • Adaptive Intensity: 0 (let the algorithm adjust naturally)

Fix 4: Verify VARactive_keepalives in astguiclient.conf

grep VARactive_keepalives /etc/astguiclient.conf
# Should contain at minimum: 123456
# Each digit enables a process:
# 1 = AST_update
# 2 = AST_send_listen
# 3 = AST_VDauto_dial
# 4 = AST_VDremote_agents
# 5 = AST_VDadapt
# 6 = FastAGI server

If 3 is missing, the auto-dialer will not start. If 5 is missing, adaptive dialing will not work.

Prevention

• Monitor vicidial_auto_calls count as part of your daily health checks
• Set up alerts for when the keepalive process dies
• After any server restart, verify all processes are running: ps aux | grep AST_VD

Verify It Worked

# After fixes, verify the dialer is working:
# 1. Check processes
ps aux | grep -c AST_VDauto_dial
# Should return at least 1

# 2. Watch calls being placed in real-time
asterisk -rx "core show channels" | tail -5

# 3. Check vicidial_auto_calls is being used (not stuck)
mysql -u cron -p1234 asterisk -e \
  "SELECT COUNT(*), MAX(call_time) FROM vicidial_auto_calls;"

---

10. WebRTC / ViciPhone Not Working

Symptoms

• ViciPhone shows "conn.failed" or "WebSocket Connection Failed".
• The phone icon appears but calls cannot be placed or received.
• Browser console shows SSL/certificate errors.
• "No one is in your session" error after login.
• ViciPhone connects but there is no audio.

Root Cause

ViciPhone (WebRTC) requires a precise chain of components to work:

1. Valid SSL certificate — Self-signed certificates DO NOT work with WebRTC in modern browsers
2. Asterisk WebSocket listener on port 8089 — Must be configured in http.conf
3. PJSIP WebSocket transport — res_pjsip_transport_websocket.so must be loaded
4. Phone template with WebRTC settings — DTLS, encryption, ICE support
5. Firewall allowing port 8089 — Both TCP and TLS
6. STUN server configuration — For NAT traversal of WebRTC media

Diagnosis Steps

1. Check SSL certificate:

   # Verify certificate exists and is valid
   openssl x509 -in /etc/letsencrypt/live/YOUR_DOMAIN/fullchain.pem -text -noout | grep "Not After"

   # Test HTTPS connectivity
   curl -v https://YOUR_DOMAIN 2>&1 | grep "SSL certificate"

1. Check Asterisk HTTP/WebSocket listener:

   asterisk -rx "http show status"
   # Should show port 8089 with TLS enabled

   # Test WebSocket connectivity from outside:
   curl -v -k https://YOUR_DOMAIN:8089/ws 2>&1 | head -20

1. Check PJSIP WebSocket module:

   asterisk -rx "module show like websocket"
   # Must show:
   # res_pjsip_transport_websocket.so
   # res_http_websocket.so

If missing:

   asterisk -rx "module load res_http_websocket.so"
   asterisk -rx "module load res_pjsip_transport_websocket.so"

1. Check firewall:

   # Port 8089 must be open
   ss -tlnp | grep 8089
   # Or:
   netstat -tlnp | grep 8089

1. Check browser console (F12 → Console tab):
   • Look for WebSocket errors, SSL errors, or OICQ errors
   • "Mixed Content" errors mean you are loading HTTP resources on an HTTPS page

Solution

Step 1: Obtain a valid SSL certificate

# Install certbot
# ViciBox/openSUSE:
zypper install certbot
# AlmaLinux:
dnf install certbot

# Get a certificate (stop Apache temporarily)
systemctl stop apache2  # or httpd
certbot certonly --standalone -d dialer.yourdomain.com
systemctl start apache2  # or httpd

# Certificate files will be in:
# /etc/letsencrypt/live/dialer.yourdomain.com/fullchain.pem
# /etc/letsencrypt/live/dialer.yourdomain.com/privkey.pem

Copy certs to Asterisk:

mkdir -p /etc/asterisk/keys
cp /etc/letsencrypt/live/dialer.yourdomain.com/fullchain.pem /etc/asterisk/keys/
cp /etc/letsencrypt/live/dialer.yourdomain.com/privkey.pem /etc/asterisk/keys/
chown asterisk:asterisk /etc/asterisk/keys/*

Set up auto-renewal:

# Add to cron:
0 3 * * 1 certbot renew --quiet --deploy-hook "cp /etc/letsencrypt/live/dialer.yourdomain.com/*.pem /etc/asterisk/keys/ && asterisk -rx 'module reload res_http_websocket.so'"

Step 2: Configure Asterisk http.conf

Edit /etc/asterisk/http.conf:

[general]
enabled=yes
bindaddr=0.0.0.0
bindport=8088
tlsenable=yes
tlsbindaddr=0.0.0.0:8089
tlscertfile=/etc/asterisk/keys/fullchain.pem
tlsprivatekey=/etc/asterisk/keys/privkey.pem

Reload:

asterisk -rx "module reload res_http_websocket.so"

Step 3: Configure PJSIP WebSocket transport

Edit /etc/asterisk/pjsip.conf — add the WebSocket transport:

[transport-wss]
type=transport
protocol=wss
bind=0.0.0.0

Step 4: Create a WebRTC phone template in ViciDial

Go to Admin → Phones → Templates, create a new template:

Template Contents:

type=friend
host=dynamic
encryption=yes
avpf=yes
icesupport=yes
directmedia=no
transport=wss
force_avp=yes
dtlsenable=yes
dtlsverify=no
dtlscertfile=/etc/asterisk/keys/fullchain.pem
dtlsprivatekey=/etc/asterisk/keys/privkey.pem
dtlssetup=actpass
rtcp_mux=yes

Step 5: Configure ViciDial Server settings

Go to Admin → Servers → [Your Server]:

• Web Socket URL: wss://dialer.yourdomain.com:8089/ws
• External Server IP: your server's public IP or FQDN

Go to Admin → System Settings:

• WebPhone URL: leave default or set to your HTTPS URL
• WebPhone Auto-Answer: Y (recommended)

Step 6: Open firewall port

# firewalld (AlmaLinux):
firewall-cmd --permanent --add-port=8089/tcp
firewall-cmd --reload

# iptables (ViciBox):
iptables -A INPUT -p tcp --dport 8089 -j ACCEPT

Prevention

• Set up certificate auto-renewal before the cert expires (90 days for Let's Encrypt)
• Test ViciPhone after every Asterisk update
• Keep browser updated — WebRTC standards evolve frequently

Verify It Worked

# Check WebSocket is listening
asterisk -rx "http show status"
# Should show: "HTTPS Server Enabled and Bound to 0.0.0.0:8089"

# Test from browser: open developer tools (F12), Console, type:
# var ws = new WebSocket('wss://dialer.yourdomain.com:8089/ws');
# ws.onopen = function() { console.log('Connected!'); };
# Should print "Connected!" if working

---

11. Chrome JS Throttling Deep Dive

Symptoms

This is a deeper technical exploration of Problem #1. Read this section if you need to understand the exact mechanism and deploy organization-wide fixes.

Timeline of Chrome Throttling Changes

Date	Chrome Version	Change	ViciDial Impact
Mar 2017	v57	Background tabs: setTimeout/setInterval limited to max 1 wake-up/second	Minimal — ViciDial's 1-second timer still works within limit
Oct 2019	v78	Chained timers (5+ chains of setTimeout) further throttled	Occasional missed updates after long background periods
Jan 2021	v88	Intensive Wake-Up Throttling: after 5 min hidden + 30s silent + no WebRTC → 1 wake-up/minute	Critical — Agent screen stops updating, agent goes LAGGED
Apr 2021	v90	chrome://flags#intensive-wake-up-throttling flag removed	Cannot disable via flags anymore, only enterprise policy
2022+	v100+	Throttling behavior stable, no further changes	Same as v88+ behavior
2024+	v120+	Additional "Energy Saver" mode further reduces background activity	Can worsen the problem on laptops

How Chrome Decides to Throttle

Chrome applies intensive throttling when ALL of these conditions are met simultaneously:

1. The page has been hidden (in a background tab) for more than 5 minutes
2. The timer chain count is 5 or greater (setTimeout calling setTimeout)
3. The page has been silent (no audio playing) for at least 30 seconds
4. WebRTC is NOT in use on the page
5. The page has not called navigator.locks.request() with a Web Lock

If ANY one of these conditions is false, intensive throttling does not apply. This is why the ViciDial workarounds work — they break condition 3 (play sound) or condition 4 (use WebRTC) or bypass timers entirely (EventSource).

ViciDial-Specific Workarounds in Detail

Agent Screen Timer: EventSource (SVN 3407+)

This is the most elegant fix. Instead of JavaScript timers (which Chrome throttles), the server pushes events to the browser via Server-Sent Events:

Browser opens: /agc/vdc_script_eventsource.php
Server sends: data: {"time": "1234567890"}\n\n  (every 1 second)
Browser receives event → triggers screen refresh

Server-Sent Events (EventSource) are NOT subject to JavaScript timer throttling because they are a network connection, not a timer. Chrome maintains the connection even in background tabs.

Configuration: Admin → System Settings → Agent Screen Timer = EventSource

Agent Hidden Browser Sound (SVN 3390+)

This breaks condition 3 (silence) by playing a tiny audio file every N seconds:

Configuration:

• Admin → System Settings → Agent Hidden Browser Sound = Y
• Admin → System Settings → Agent Hidden Browser Sound Seconds = 20

The system plays a near-silent audio snippet (an almost-inaudible tone) via the Web Audio API when it detects the page is hidden. Chrome classifies the tab as "audible" and exempts it from intensive throttling.

Chrome Enterprise Policy Deployment

For organizations managing Chrome via GPO or MDM:

Windows GPO:

Computer Configuration → Administrative Templates → Google → Google Chrome
→ "Intensive wake up throttling enabled" → Disabled

Registry equivalent:

[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Google\Chrome]
"IntensiveWakeUpThrottlingEnabled"=dword:00000000

Linux managed Chrome:

mkdir -p /etc/opt/chrome/policies/managed
cat > /etc/opt/chrome/policies/managed/vicidial-throttle.json << 'EOF'
{
  "IntensiveWakeUpThrottlingEnabled": false
}
EOF

macOS managed Chrome:

defaults write com.google.Chrome IntensiveWakeUpThrottlingEnabled -bool false

Firefox and Edge Behavior

• Firefox: Throttles background tabs to 1 wake-up/second (same as Chrome v57). Does NOT implement intensive 1/minute throttling. ViciDial works in Firefox background tabs without any workaround.
• Microsoft Edge: Uses Chromium engine — identical throttling behavior to Chrome. Same workarounds apply. Edge enterprise policy: IntensiveWakeUpThrottlingEnabled in Edge policy templates.
• Safari: Has its own throttling mechanism but is rarely used for ViciDial agent desktops.

Prevention

Deploy the EventSource timer method on every ViciDial installation regardless of browser policy, as it is the most reliable long-term fix. Browser policies can be deprecated without notice, but EventSource is a fundamental web standard.

---

12. After-Hours / No-Agent Call Routing

Symptoms

• Inbound calls during off-hours are silently dropped (caller hears nothing, then disconnect).
• Calls come in when no agents are logged in and get hung up immediately.
• Callers report hearing silence instead of a voicemail prompt or after-hours message.
• The CDR shows disposition AFTHRS (after hours) or NANQUE (no agent, no queue).

Root Cause

ViciDial in-groups have several settings that control what happens when calls arrive outside business hours or when no agents are available. The most dangerous misconfiguration is setting any action to HANGUP, which silently drops the call with no audio feedback to the caller.

Key in-group fields:

• After Hours Action — What to do when a call arrives outside the in-group's call time
• After Hours Message Filename — Audio file to play before the after-hours action
• No Agent No Queue — Whether to queue calls when no agents are logged in
• No Agent No Queue Action — What to do when no agents are available and queueing is disabled
• Drop Action — What to do when a call has been waiting longer than Drop Call Seconds

Diagnosis Steps

1. Check in-group settings:

   SELECT group_id, group_name,
          after_hours_action, after_hours_message_filename,
          after_hours_exten, after_hours_voicemail,
          no_agent_no_queue, no_agent_no_queue_action,
          drop_action, drop_exten
   FROM vicidial_inbound_groups
   WHERE group_id = 'YOUR_INGROUP';

1. Check the call time assigned to the in-group:

   SELECT group_id, call_time_id
   FROM vicidial_inbound_groups
   WHERE group_id = 'YOUR_INGROUP';

   -- Then check the call time definition:
   SELECT * FROM vicidial_call_times
   WHERE call_time_id = 'THE_CALL_TIME_ID';

1. Check recent dropped/after-hours calls:

   SELECT call_date, phone_number, status, term_reason, queue_seconds
   FROM vicidial_closer_log
   WHERE campaign_id = 'YOUR_INGROUP'
     AND status IN ('AFTHRS', 'NANQUE', 'DROP', 'HANGUPCL')
     AND call_date > NOW() - INTERVAL 24 HOUR
   ORDER BY call_date DESC;

Solution

Safe after-hours configuration:

Go to Admin → In-Groups → [Your In-Group] → Modify:

Setting	Safe Value	Dangerous Value	Description
After Hours Action	MESSAGE or EXTEN or VOICEMAIL or IN_GROUP	HANGUP	What to do outside business hours
After Hours Message	(select an audio file)	(blank)	Audio file to play to the caller
After Hours Exten	A ring group extension (e.g., 8300)	(blank)	Extension to transfer to
No Agent No Queue	NO_AGENT_NO_QUEUE or N	—	Whether to queue when no agents
No Agent No Queue Action	MESSAGE or EXTEN or VOICEMAIL	HANGUP	What to do when no agents and no queue
Drop Action	MESSAGE or VOICEMAIL or IN_GROUP	HANGUP	What to do when wait time exceeds limit

Example: Route after-hours calls to a ring group

After Hours Action: EXTEN
After Hours Exten: 8300
After Hours Context: default

Where extension 8300 is defined in /etc/asterisk/extensions_custom.conf as a ring group that rings Zoiper softphones or mobile numbers:

[default]
exten => 8300,1,NoOp(After-hours ring group)
exten => 8300,n,Dial(SIP/phone1&SIP/phone2&SIP/phone3,30)
exten => 8300,n,VoiceMail(8300@default,u)
exten => 8300,n,Hangup()

Example: Play a message and go to voicemail

After Hours Action: VOICEMAIL
After Hours Message: after_hours_greeting    (audio file uploaded via Admin → Audio Store)
After Hours Voicemail: 8300                  (voicemail box number)

WARNING

NEVER set Drop Action, After Hours Action, or No Agent No Queue Action to HANGUP in production. This silently drops calls with no feedback to the caller. Callers will hear silence and then a disconnect — they will not know why and will not call back. Use MESSAGE, VOICEMAIL, EXTEN, or IN_GROUP instead.

If you need to reject calls, at minimum play a message: "We are currently closed. Please call back during business hours." Then route to voicemail or hang up after the message plays.

Prevention

• Audit all in-groups quarterly for HANGUP actions
• Test after-hours routing by temporarily changing the call time to a past window
• Monitor AFTHRS and NANQUE dispositions daily

Verify It Worked

# Set the in-group call time temporarily to exclude the current hour
# Make a test call
# Verify the caller hears the after-hours message and is routed correctly
# Reset the call time

# Check the closer_log for the test call:
mysql -u cron -p1234 asterisk -e \
  "SELECT call_date, phone_number, status, term_reason \
   FROM vicidial_closer_log \
   WHERE campaign_id='YOUR_INGROUP' \
   ORDER BY call_date DESC LIMIT 5;"

---

13. Transfer / Conference Problems

Symptoms

• Agent clicks transfer and sees "XFER LINE HUNG UP" immediately.
• Three-way conference connects but the transferred party drops after a few seconds.
• Blind transfers ring but the call disconnects when answered.
• The "Hangup Xfer Line" button is grayed out and the agent cannot disconnect the transfer leg.
• Warm/consultative transfers work for internal extensions but fail for external numbers.

Root Cause

Transfer issues in ViciDial typically fall into these categories:

1. Missing 3-Way Call Prefix — The campaign does not have the correct dial prefix configured for 3-way calls, so external transfers fail to route through the trunk.
2. Conference resource limits — ViciDial uses MeetMe/ConfBridge conferences for transfers. If conference slots are exhausted, transfers fail.
3. Trunk routing — The transfer dials a number but it goes to the wrong context or trunk.
4. Timeout settings — The transfer leg times out before the recipient answers.

Diagnosis Steps

1. Check the campaign 3-Way settings:

   SELECT campaign_id, three_way_call_cid, three_way_dial_prefix,
          xfer_park_hide_number
   FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

1. Check the agent's Asterisk console during a transfer attempt:

   asterisk -rvvv
   # Then have the agent attempt a transfer — watch for errors

1. Check conference resources:

   -- How many conferences are allocated?
   SELECT server_ip, COUNT(*) AS conf_count
   FROM vicidial_conferences
   GROUP BY server_ip;

   -- How many are currently in use?
   SELECT server_ip, COUNT(*) AS active_confs
   FROM vicidial_conferences
   WHERE leave_3way_datetime IS NOT NULL
   GROUP BY server_ip;

1. Check for stuck conference entries:

   SELECT conf_exten, extension, server_ip, leave_3way_datetime
   FROM vicidial_conferences
   WHERE leave_3way_datetime < NOW() - INTERVAL 1 HOUR
     AND leave_3way_datetime IS NOT NULL;

Solution

Fix 1: Set the 3-Way Dial Prefix

If your outbound trunk requires a prefix (e.g., 9 for an outside line), the same prefix must be set for 3-way calls:

Admin → Campaigns → [Your Campaign] → Detail View:

• 3-Way Call Dial Prefix: Set to your dial prefix (e.g., 9)
• This is a separate field from the normal Dial Prefix

Fix 2: Clean up stuck conferences

-- Reset stuck conference entries
UPDATE vicidial_conferences
SET extension = '', leave_3way_datetime = NULL
WHERE leave_3way_datetime < NOW() - INTERVAL 2 HOUR;

Fix 3: Increase conference resources

If you consistently run out of conference slots:

-- Check current allocation
SELECT MIN(conf_exten), MAX(conf_exten), COUNT(*)
FROM vicidial_conferences
WHERE server_ip = 'YOUR_SERVER_IP';

You may need to regenerate conference extensions via:
Admin → Servers → [Your Server] → Conference Table Re-Generate

Transfer types explained

• Blind Transfer: Agent clicks the transfer button and selects "Blind Transfer" — the call is immediately sent to the destination without the agent staying on the line. The agent is freed immediately.
• Warm/Consultative Transfer: Agent clicks the transfer button and dials the destination. All three parties (agent, customer, destination) are in a conference. The agent can introduce the customer, then click "Hangup Xfer Line" or "Leave 3-Way" to drop from the conference.
• Park Call: The customer is placed on hold while the agent dials the destination. Once connected, the agent can conference the customer in.

Prevention

• Always test transfers immediately after setting up a new campaign
• Include the 3-Way Dial Prefix in your campaign setup checklist
• Monitor conference resource usage during peak hours

Verify It Worked

Have an agent make a test call and attempt each transfer type. The Asterisk CLI (asterisk -rvvv) should show the INVITE going out to the transfer destination without errors.

---

14. Callback Scheduling Issues

Symptoms

• Agent sets a callback but it never fires — the lead is never re-dialed at the scheduled time.
• Callbacks appear in the agent's callback list but the customer is not called.
• USERONLY callbacks work but ANYONE callbacks do not (or vice versa).
• Lead status shows CBHOLD permanently and never changes to CALLBK.

Root Cause

ViciDial callbacks work through a specific mechanism:

1. Agent dispositions a call as CALLBK → a popup appears to set date/time and USERONLY flag
2. The lead status is set to CBHOLD in vicidial_list and a record is created in vicidial_callbacks
3. At the scheduled time, AST_VDhopper.pl checks for due callbacks:
   • ANYONE callbacks: The lead status is changed from CBHOLD to CALLBK, making it eligible for the hopper if CALLBK is in the campaign's dial statuses
   • USERONLY callbacks: The callback appears in the specific agent's callback list when they log in

Common failure points:

1. CALLBK is not in the campaign's Dial Statuses list
2. AST_VDhopper.pl is not running (so CBHOLD never becomes CALLBK)
3. The callback date/time is in the lead's local timezone but the agent set it in their own timezone
4. The agent who set a USERONLY callback is not logged in when the callback is due
5. The list containing the callback lead is inactive

Diagnosis Steps

1. Check for pending callbacks:

   SELECT cb.callback_id, cb.lead_id, cb.campaign_id, cb.status, cb.user,
          cb.callback_time, cb.recipient, cb.comments,
          vl.status AS lead_status, vl.list_id
   FROM vicidial_callbacks cb
   JOIN vicidial_list vl ON cb.lead_id = vl.lead_id
   WHERE cb.status = 'ACTIVE'
   ORDER BY cb.callback_time;

1. Check if CALLBK is a dial status for the campaign:

   SELECT campaign_id, dial_statuses
   FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';
   -- The output must contain 'CALLBK' in the dial_statuses string

1. Check that AST_VDhopper.pl is running:

   ps aux | grep AST_VDhopper

1. Check the lead status:

   SELECT lead_id, status, called_since_last_reset, list_id
   FROM vicidial_list
   WHERE lead_id = THE_LEAD_ID;

If status is still CBHOLD and the callback time has passed, the hopper script is not processing it.

Solution

Fix 1: Add CALLBK to dial statuses

Admin → Campaigns → [Your Campaign] → Detail View → Dial Statuses:
Check the box for CALLBK.

Or via SQL:

UPDATE vicidial_campaigns
SET dial_statuses = CONCAT(dial_statuses, ' CALLBK')
WHERE campaign_id = 'YOUR_CAMPAIGN'
  AND dial_statuses NOT LIKE '%CALLBK%';

Fix 2: Ensure AST_VDhopper.pl is running

# It should be in cron running every minute:
crontab -l | grep VDhopper
# Must show: * * * * * /usr/share/astguiclient/AST_VDhopper.pl -q

# If missing, add it

Fix 3: Manually trigger a stuck callback

-- Change the lead status from CBHOLD to CALLBK
UPDATE vicidial_list SET status = 'CALLBK'
WHERE lead_id = THE_LEAD_ID AND status = 'CBHOLD';

-- Update the callback record
UPDATE vicidial_callbacks SET status = 'LIVE'
WHERE lead_id = THE_LEAD_ID AND status = 'ACTIVE';

USERONLY vs ANYONE Callbacks

Type	How it works	When it fires
ANYONE	Lead goes into the hopper for any agent to dial	When callback_time passes AND AST_VDhopper.pl runs AND CALLBK is a dial status
USERONLY	Lead appears ONLY in the specific agent's callback list	When the agent logs in, they see a count next to "Callbacks" — they click it and manually select which callback to dial

USERONLY callbacks are NOT automatically dialed. The agent must click the Callbacks link and choose the lead. This is by design — the customer requested to speak with that specific agent.

Prevention

• Always include CALLBK in the Dial Statuses for any campaign that uses callbacks
• Train agents on the difference between USERONLY and ANYONE callbacks
• Monitor the vicidial_callbacks table for stuck ACTIVE records with past dates

Verify It Worked

-- Check for callbacks that should have fired but have not
SELECT COUNT(*) AS overdue_callbacks
FROM vicidial_callbacks
WHERE status = 'ACTIVE'
  AND callback_time < NOW() - INTERVAL 1 HOUR;
-- This should be 0 or very low

---

15. Time Synchronization Errors

Symptoms

• Agents see: "There is a time synchronization problem with your system, please tell your system administrator" when trying to log in.
• Agent sessions randomly disconnect with time-related errors.
• Report timestamps are wrong.
• Callbacks fire at the wrong time.

Root Cause

ViciDial compares the server's system time, PHP's time, and the database time. If these differ by more than a few seconds, ViciDial raises a synchronization error. Common causes:

1. NTP not configured or not running — Server clock drifts
2. Server IP changed — The server_ip in astguiclient.conf or system_settings does not match
3. Timezone mismatch — PHP timezone differs from system timezone or database timezone
4. VM clock drift — Virtual machines are notorious for clock drift, especially on overloaded hosts

Diagnosis Steps

1. Compare system time, PHP time, and database time:

   # System time
   date

   # PHP time
   php -r "echo date('Y-m-d H:i:s') . PHP_EOL;"

   # Database time
   mysql -u cron -p1234 -e "SELECT NOW();"

All three should be within 1 second of each other.

1. Check NTP status:

   # ViciBox/openSUSE (chrony):
   chronyc tracking
   # Or older systems (ntpd):
   ntpq -p

   # AlmaLinux (chrony):
   chronyc tracking

1. Check timezone configuration:

   # System timezone
   timedatectl

   # PHP timezone
   php -r "echo ini_get('date.timezone') . PHP_EOL;"

   # MySQL timezone
   mysql -u cron -p1234 -e "SELECT @@global.time_zone, @@session.time_zone;"

1. Check astguiclient.conf server_ip:

   grep VARserver_ip /etc/astguiclient.conf
   # Must match the actual IP of this server

Solution

Fix 1: Configure NTP

# ViciBox/openSUSE — install and enable chrony:
zypper install chrony
systemctl enable chronyd
systemctl start chronyd

# AlmaLinux:
dnf install chrony
systemctl enable chronyd
systemctl start chronyd

# Force immediate sync:
chronyc makestep

# Verify:
chronyc tracking

Fix 2: Set consistent timezone

# Set system timezone (example: US Eastern)
timedatectl set-timezone America/New_York

# Set PHP timezone in /etc/php.ini or /etc/php7/apache2/php.ini:
# date.timezone = America/New_York

# Set MySQL timezone in /etc/my.cnf:
# [mysqld]
# default-time-zone = 'America/New_York'

# Restart services:
systemctl restart apache2  # or httpd
systemctl restart mariadb

Fix 3: Update server_ip after IP change

If you changed the server's IP address, you must run the update script:

/usr/share/astguiclient/ADMIN_update_server_ip.pl --old-server_ip=OLD_IP_ADDRESS

Follow the on-screen prompts, then reboot the server.

Also verify in the database:

SELECT server_ip FROM system_settings;
SELECT server_ip, active_asterisk_server FROM servers;

Fix 4: Fix VM clock drift

For VMs, install and enable the VM guest agent:

# VMware:
zypper install open-vm-tools  # or dnf install open-vm-tools
systemctl enable vmtoolsd

# KVM/Proxmox:
zypper install qemu-guest-agent  # or dnf install qemu-guest-agent
systemctl enable qemu-guest-agent

Prevention

• Always configure NTP during initial server setup
• Include timezone configuration in your installation checklist
• Monitor NTP sync status in your monitoring system

Verify It Worked

# All three times should match:
echo "System: $(date '+%Y-%m-%d %H:%M:%S')"
echo "PHP:    $(php -r "echo date('Y-m-d H:i:s');")"
echo "MySQL:  $(mysql -u cron -p1234 -Nse "SELECT NOW();")"

---

16. AMD (Answering Machine Detection) Tuning

Symptoms

• AMD detects too many live humans as answering machines (false positives) — agents miss real calls.
• AMD fails to detect obvious answering machines (false negatives) — agents waste time listening to voicemail greetings.
• Calls are dropped or dispositioned as AA (Answering Machine Auto) incorrectly.
• Long silence before the agent connects (AMD analysis delay).

Root Cause

AMD works by analyzing audio characteristics of the first few seconds after a call is answered:

• Humans typically say a short greeting ("Hello?") followed by silence (waiting for response)
• Answering machines typically play a longer greeting without pausing

The accuracy of AMD depends heavily on the amd.conf parameters and the characteristics of the phone network. Out of the box, AMD is approximately 65% accurate. With careful tuning, 75-80% accuracy is achievable. 100% accuracy is not possible.

Key Parameters in /etc/asterisk/amd.conf

[general]
; Maximum initial silence before a greeting starts.
; If exceeded → detected as MACHINE (assumes voicemail "beep waiting")
initial_silence = 2500

; Maximum length of a greeting.
; If exceeded → detected as MACHINE (long greeting = answering machine)
greeting = 1500

; Silence after the greeting ends.
; Must be this long to determine the greeting is over.
after_greeting_silence = 800

; Maximum time the algorithm will analyze before giving up.
; After this time, the call is sent to the agent regardless.
total_analysis_time = 5000

; Minimum duration of sound to be considered a "word"
min_word_length = 100

; Silence between words to consider the next sound a new word
between_words_silence = 50

; Maximum words before declaring MACHINE
; Humans say 1-2 words ("Hello?"), machines say many words
maximum_number_of_words = 3

; Audio level threshold below which is considered silence (0-32767)
silence_threshold = 256

Diagnosis Steps

1. Check current AMD settings:

   cat /etc/asterisk/amd.conf

1. Check campaign AMD configuration:

   SELECT campaign_id, cpd_amd_action, amd_send_to_vmx,
          waitforsilence_options
   FROM vicidial_campaigns
   WHERE campaign_id = 'YOUR_CAMPAIGN';

cpd_amd_action values:

• DISABLED — AMD is off
• DISPO — AMD detects machine → call dispositioned as AA and hung up
• MESSAGE — AMD detects machine → play the campaign's answering machine message (voicemail drop)

1. Check AMD accuracy from recent calls:

   -- Count AMD dispositions vs agent-assigned dispositions
   SELECT status, COUNT(*) AS cnt
   FROM vicidial_log
   WHERE campaign_id = 'YOUR_CAMPAIGN'
     AND call_date > NOW() - INTERVAL 24 HOUR
     AND status IN ('AA', 'AM', 'A', 'NA', 'SALE', 'NI')
   GROUP BY status;

Compare the AA (auto AMD) count to what agents report. If agents say they are getting answering machines that were NOT caught by AMD, false negatives are high.

Solution

Recommended amd.conf settings for ViciDial

These settings balance accuracy with minimal false positives:

[general]
initial_silence = 2500
greeting = 1500
after_greeting_silence = 800
total_analysis_time = 5000
min_word_length = 100
between_words_silence = 50
maximum_number_of_words = 3
silence_threshold = 256

Tuning guidelines

Problem	Parameter to Adjust	Direction
Too many false positives (humans detected as machines)	Increase greeting	1500 → 2000
Too many false positives	Increase maximum_number_of_words	3 → 5
Too many false negatives (machines not detected)	Decrease greeting	1500 → 1200
Too many false negatives	Decrease maximum_number_of_words	3 → 2
Long delay before agent connects	Decrease total_analysis_time	5000 → 3500
Noisy phone lines causing false silence detection	Increase silence_threshold	256 → 512

Enable AMD on a campaign

Admin → Campaigns → [Your Campaign] → Detail View:

• CPD AMD Action: DISPO (to just hang up on machines) or MESSAGE (to leave a voicemail)
• AMD Send Notification: Y (to notify agents about AMD results)
• If using MESSAGE: Upload an audio file via Admin → Audio Store, then set it as the Answering Machine Message for the campaign

Apply changes

asterisk -rx "module reload app_amd.so"

Prevention

• Start with conservative settings (higher greeting, higher maximum_number_of_words) and tighten gradually
• Run test batches of 500+ calls and analyze the AMD hit rate before deploying to production
• Different carriers and countries have different audio characteristics — tune per trunk/campaign
• Keep a list of known answering machine numbers in your test list to validate AMD accuracy

Verify It Worked

-- Monitor AMD accuracy over 24 hours
SELECT
  COUNT(CASE WHEN status = 'AA' THEN 1 END) AS amd_detected,
  COUNT(CASE WHEN status NOT IN ('AA','AM') THEN 1 END) AS agent_handled,
  COUNT(*) AS total_calls
FROM vicidial_log
WHERE campaign_id = 'YOUR_CAMPAIGN'
  AND call_date > NOW() - INTERVAL 24 HOUR;

---

17. Cron Jobs / Keepalive Dying

Symptoms

• ViciDial stops functioning — no calls placed, agent screens freeze.
• ps aux | grep astguiclient shows no running ViciDial processes.
• The Real-Time report shows data but nothing moves.
• After a server reboot, ViciDial does not start automatically.

Root Cause

ViciDial relies entirely on Perl scripts running via cron. If the crontab is empty, corrupted, or if the cron daemon itself is not running, ViciDial stops functioning completely.

The most critical script is ADMIN_keepalive_ALL.pl, which monitors and restarts all other ViciDial processes. If this one script is running, it will keep everything else alive.

Complete Required Crontab Listing

Below is the complete crontab for a standard ViciDial server. Every line matters:

### Recording processing — mix, compress, FTP
0,3,6,9,12,15,18,21,24,27,30,33,36,39,42,45,48,51,54,57 * * * * /usr/share/astguiclient/AST_CRON_audio_1_move_mix.pl
1,4,7,10,13,16,19,22,25,28,31,34,37,40,43,46,49,52,55,58 * * * * /usr/share/astguiclient/AST_CRON_audio_2_compress.pl --MP3
2,5,8,11,14,17,20,23,26,29,32,35,38,41,44,47,50,53,56,59 * * * * /usr/share/astguiclient/AST_CRON_audio_3_ftp.pl --MP3

### CRITICAL: Keepalive for all astguiclient processes (runs every minute)
* * * * * /usr/share/astguiclient/ADMIN_keepalive_ALL.pl

### Kill hung Asterisk updater processes
* * * * * /usr/share/astguiclient/AST_manager_kill_hung_congested.pl

### Voicemail updater
* * * * * /usr/share/astguiclient/AST_vm_update.pl

### Conference validator
* * * * * /usr/share/astguiclient/AST_conf_update.pl

### VICIDIAL hopper updater (loads leads into the dial hopper)
* * * * * /usr/share/astguiclient/AST_VDhopper.pl -q

### Agent log cleanup (removes stale entries)
33 * * * * /usr/share/astguiclient/AST_cleanup_agent_log.pl

### Flush queue DB table
0 * * * * /usr/share/astguiclient/AST_flush_DBqueue.pl

### GMT offset adjustment for leads
1 1,7 * * * /usr/share/astguiclient/ADMIN_adjust_GMTnow_on_leads.pl --debug --list-settings

### Database optimization (table maintenance)
1 3 * * * /usr/share/astguiclient/AST_DB_optimize.pl

### Daily log archiving (adjust --days as needed)
3 1 * * * /usr/share/astguiclient/ADMIN_archive_log_tables.pl --daily --closer-log

### Clean old astguiclient and Asterisk logs
28 0 * * * find /var/log/astguiclient/ -type f -mtime +2 -delete
29 0 * * * find /var/log/asterisk/ -type f -mtime +7 -delete

Diagnosis Steps

1. Check if cron daemon is running:

   systemctl status cron    # Debian/Ubuntu
   systemctl status crond   # RHEL/AlmaLinux/ViciBox

1. Check crontab contents:

   crontab -l

If this is empty, the crontab was wiped (common after accidental crontab -r).

1. Check if ViciDial processes are running:

   ps aux | grep -E "(AST_VD|ADMIN_keep|AST_update|AST_send)" | grep -v grep

1. Check the keepalive log:

   ls -la /var/log/astguiclient/
   tail -50 /var/log/astguiclient/keepalive.log

1. Run keepalive manually to see what it starts:

   /usr/share/astguiclient/ADMIN_keepalive_ALL.pl --debugX

Solution

Fix 1: Restore the crontab

If the crontab is empty or missing, paste the complete crontab listing above into:

crontab -e

Fix 2: Ensure cron daemon is running and enabled

# ViciBox/openSUSE:
systemctl enable crond
systemctl start crond

# AlmaLinux:
systemctl enable crond
systemctl start crond

Fix 3: Check VARactive_keepalives

grep VARactive_keepalives /etc/astguiclient.conf

This should contain a string like 1234568 where each digit enables a process:

Digit	Process	Description
1	AST_update_realtime	Real-time data updater
2	AST_send_listen	Send/listen to Asterisk manager
3	AST_VDauto_dial	Predictive dialer
4	AST_VDremote_agents	Remote agent handling
5	AST_VDadapt	Adaptive dialing algorithm
6	FastAGI_log	FastAGI server for AGI scripts
7	AST_VDauto_dial_FILL	Fill dialing (secondary campaigns)
8	ip_relay	IP relay for remote agents
9	Timeclock	Employee timeclock

A minimum configuration is 123456. If any needed digit is missing, that process will not be started by the keepalive.

Prevention

• Back up your crontab: crontab -l > /root/crontab-backup-$(date +%Y%m%d).txt
• Never use crontab -r (removes entire crontab). Use crontab -e to edit.
• Monitor the keepalive process — if it stops, everything stops

Verify It Worked

# After restoring crontab, wait 2 minutes, then check all processes:
ps aux | grep -c "AST_VD"
# Should return 3+ (VDhopper, VDauto_dial, VDadapt at minimum)

ps aux | grep -c "ADMIN_keep"
# Should return 1+

---

18. DAHDI / Timing Source Problems

Symptoms

• Agent conferences fail — the agent logs in but cannot be connected to calls.
• MeetMe conferences do not work: "Unable to open DAHDI pseudo channel: Cannot allocate memory"
• Asterisk log shows: "dahdi: Detected time shift" or "No timing interfaces available"
• Music on Hold plays garbled or not at all.
• Call transfers fail silently.

Root Cause

ViciDial uses MeetMe conferences (older versions) or ConfBridge (newer versions) for the agent-to-caller audio bridge. MeetMe requires DAHDI timing modules, even on systems with no physical telephony hardware.

In older DAHDI versions, the dahdi_dummy module provided a software-based timing source. Since DAHDI Linux 2.3.0, dahdi_dummy was merged into the core dahdi module.

Common issues:

1. DAHDI kernel module not loaded after a kernel update
2. Running in a VM without proper timer emulation
3. DAHDI modules need to be rebuilt after a kernel update (ViciBox)
4. Timer exhaustion — too many conferences consuming all pseudo channels

Diagnosis Steps

1. Check if DAHDI is loaded:

   lsmod | grep dahdi
   # Should show dahdi and possibly dahdi_dummy (on older systems)

1. Check DAHDI timing:

   dahdi_test
   # Should show 99.5%+ accuracy
   # If it shows "No DAHDI devices found", the module is not loaded

1. Check Asterisk timer availability:

   asterisk -rx "timing test"
   # Should return timing source and accuracy

1. Check for DAHDI errors in logs:

   dmesg | grep -i dahdi
   journalctl -u dahdi | tail -20

1. Check MeetMe/ConfBridge status:

   asterisk -rx "meetme list"
   # Or for newer Asterisk:
   asterisk -rx "confbridge list"

Solution

Fix 1: Load DAHDI modules

# Load the modules
modprobe dahdi
modprobe dahdi_dummy  # Only needed for DAHDI < 2.3.0

# Verify
lsmod | grep dahdi

# Configure DAHDI
dahdi_cfg -v

# Restart Asterisk
systemctl restart asterisk

Fix 2: Rebuild DAHDI after kernel update (ViciBox)

# ViciBox provides a helper script:
vicibox-dahdi-install

# Or manually rebuild:
cd /usr/src/dahdi-linux-*
make clean
make
make install
dahdi_cfg -v
modprobe dahdi
systemctl restart asterisk

Fix 3: Persist DAHDI modules across reboots

# Create a modules-load file
echo "dahdi" > /etc/modules-load.d/dahdi.conf
# On older systems, add to /etc/modules:
echo "dahdi" >> /etc/modules

Fix 4: Use ConfBridge instead of MeetMe (Asterisk 13+)

ConfBridge does NOT require DAHDI — it uses Asterisk's internal timing. If you are on Asterisk 13+, you can switch to ConfBridge:

asterisk -rx "module load app_confbridge.so"

# Check if ViciDial is configured to use ConfBridge:
grep conf_engine /etc/astguiclient.conf
# VARconf_engine should be 'CONFBRIDGE' (not 'MEETME')

If changing from MeetMe to ConfBridge, update /etc/astguiclient.conf:

VARconf_engine => CONFBRIDGE

And restart the astguiclient processes:

/usr/share/astguiclient/stop_astguiclient
/usr/share/astguiclient/start_astguiclient

Fix 5: VM timing workaround

If running in a VM and DAHDI timing is unreliable:

# Check if res_timing_timerfd is available (preferred for VMs)
asterisk -rx "module show like timing"

# Prefer timerfd over DAHDI for timing in VMs:
# In /etc/asterisk/modules.conf, ensure:
# load => res_timing_timerfd.so
# noload => res_timing_dahdi.so   (only if you don't need MeetMe)

Note: If you use MeetMe (not ConfBridge), you MUST use DAHDI timing regardless of environment. ConfBridge + res_timing_timerfd is the recommended combination for VMs.

Prevention

• After any kernel update on ViciBox, rebuild DAHDI: vicibox-dahdi-install
• On VMs, prefer ConfBridge over MeetMe
• Monitor DAHDI timing accuracy periodically
• Pin the kernel version on production servers to avoid unexpected DAHDI breakage

Verify It Worked

# DAHDI loaded and timing good:
dahdi_test
# Should show: "Best: 99.xxx% -- Loss: 0.xxx%"

# Asterisk timing working:
asterisk -rx "timing test"

# Test a conference:
asterisk -rx "meetme list"
# Or:
asterisk -rx "confbridge list"

---

19. Agent Login Failures

Symptoms

• Agent enters credentials and gets: "Login incorrect" or a blank screen.
• Agent logs in but is immediately kicked out: "No one is in your session: 8600XXX"
• Phone registration fails — agent cannot select a phone at login.
• Agent logs in but the screen freezes and never reaches READY status.

Root Cause

Agent login involves multiple components: web authentication, phone (SIP) registration, conference allocation, and session management. Failure at any point causes login issues.

1. Wrong credentials — Typo in username/password, or the user account is inactive
2. Phone not registered — The SIP phone/extension is not registered with Asterisk
3. Session conflict — Another session is already active for this user or phone
4. Server time mismatch — Time synchronization error (see Problem #15)
5. Firewall blocking agent's IP — The agent's browser cannot reach the ViciDial web interface or WebSocket
6. Apache/httpd not running — The web server is down

Diagnosis Steps

1. Check the user account:

   SELECT user, full_name, active, user_level, user_group
   FROM vicidial_users
   WHERE user = 'AGENT_USERNAME';

active must be Y and user_level must be >= 1.

1. Check if the phone is registered:

   # For SIP phones:
   asterisk -rx "sip show peers" | grep PHONE_EXTENSION
   # Status should be "OK" with a latency value

   # For PJSIP:
   asterisk -rx "pjsip show endpoints" | grep PHONE_EXTENSION

1. Check for existing sessions:

   SELECT user, server_ip, conf_exten, status, last_update_time
   FROM vicidial_live_agents
   WHERE user = 'AGENT_USERNAME';

If a stale session exists, it may block new login.

1. Check Apache/httpd is running:

   systemctl status apache2  # ViciBox/openSUSE
   systemctl status httpd    # AlmaLinux

1. Check web access from the agent's browser:

   http://YOUR_SERVER_IP/vicidial/welcome.php

If this does not load, it is a network/firewall issue.

1. Check astguiclient.conf:

   grep VARserver_ip /etc/astguiclient.conf

This must be the server's actual IP address (not 127.0.0.1 or blank).

Solution

Fix 1: Clear stale sessions

-- Remove the stale live agent entry
DELETE FROM vicidial_live_agents WHERE user = 'AGENT_USERNAME';

-- Clear conference allocations for that user
UPDATE vicidial_conferences
SET extension = '', leave_3way_datetime = NULL
WHERE extension LIKE '%AGENT_USERNAME%';

Fix 2: Re-register the phone

# Check sip-vicidial.conf for the phone entry
grep -A 5 'PHONE_EXTENSION' /etc/asterisk/sip-vicidial.conf

# If the phone entry does not exist, it needs to be created via:
# Admin → Phones → Add a New Phone

# After adding/modifying, reload SIP:
asterisk -rx "sip reload"

Fix 3: Fix session timeout

If agents are being kicked out frequently, check the web server session timeout:

# Check PHP session settings
php -i | grep session.gc_maxlifetime
# Default is 1440 seconds (24 minutes). Increase if needed:
# In /etc/php.ini or /etc/php7/apache2/php.ini:
# session.gc_maxlifetime = 86400

Also check the ViciDial login timeout in the system settings:
Admin → System Settings → Agent Login Timeout (default is 10 seconds)

Fix 4: Check for fail2ban blocks

# Check if the agent's IP is blocked by fail2ban
fail2ban-client status
fail2ban-client status apache-auth  # Check specific jail

# Unban an IP:
fail2ban-client set apache-auth unbanip AGENT_IP_ADDRESS

Fix 5: Fix blank server_ip in astguiclient.conf

# If VARserver_ip is blank or wrong:
vi /etc/astguiclient.conf
# Set: VARserver_ip => YOUR_ACTUAL_SERVER_IP

# Then restart astguiclient:
/usr/share/astguiclient/stop_astguiclient
/usr/share/astguiclient/start_astguiclient

Prevention

• Instruct agents to always log out properly (click "Logout") instead of closing the browser
• Set up a daily cleanup of stale sessions:

  -- Add to a cron script:
  DELETE FROM vicidial_live_agents
  WHERE last_update_time < NOW() - INTERVAL 8 HOUR;

• Monitor phone registrations with sip show peers

Verify It Worked

After applying fixes, have the agent:

1. Clear browser cookies and cache
2. Close all browser windows
3. Open a fresh browser tab
4. Navigate to the ViciDial agent URL
5. Log in with correct credentials

Check on the server:

asterisk -rx "sip show peers" | grep PHONE_EXT
mysql -u cron -p1234 asterisk -e \
  "SELECT user, status, last_update_time FROM vicidial_live_agents WHERE user='AGENT_USERNAME';"

---

20. Recording Playback / Archive Issues

Symptoms

• Clicking a recording link in reports gives a 404 "Not Found" error.
• Recording links show "Access Forbidden" (403 error).
• Recordings play from the local server but not from the archive server.
• The "RECORDINGS" link on the search page does not work.
• Old recordings are inaccessible after being archived to FTP.

Root Cause

Recording playback depends on Apache being configured to serve the recording files, and on the database having the correct URL for each recording. Common issues:

1. Apache alias not configured — The /RECORDINGS/ URL alias does not exist in the Apache config
2. File permissions — Apache cannot read the recording files
3. FTP archive misconfiguration — Recordings uploaded to FTP but the HTTP path in the database points to the wrong URL
4. Mixed HTTP/HTTPS — Browser blocks HTTP recording links when the admin panel is loaded via HTTPS

Diagnosis Steps

1. Check the recording URL in the database:

   SELECT recording_id, filename, location, server_ip
   FROM recording_log
   WHERE start_time > NOW() - INTERVAL 1 HOUR
   ORDER BY start_time DESC LIMIT 5;

The location field should contain a valid HTTP URL.

1. Check if the file exists on disk:

   # From the location URL, extract the filename and check:
   find /var/spool/asterisk/monitorDONE -name "FILENAME*" -type f

1. Check Apache configuration for RECORDINGS alias:

   # ViciBox/openSUSE:
   grep -r "RECORDINGS" /etc/apache2/
   # AlmaLinux:
   grep -r "RECORDINGS" /etc/httpd/

   # Or check with apache:
   apache2ctl -S 2>&1 | head -20   # ViciBox
   httpd -S 2>&1 | head -20         # AlmaLinux

1. Check file permissions:

   ls -la /var/spool/asterisk/monitorDONE/MP3/ | head -5
   # Files should be readable by the web server user (wwwrun on ViciBox, apache on AlmaLinux)

1. Test direct file access:

   curl -v http://YOUR_SERVER/RECORDINGS/MP3/test_recording.mp3

Solution

Fix 1: Configure Apache RECORDINGS alias

For ViciBox/openSUSE (/etc/apache2/conf.d/recordings.conf):

Alias /RECORDINGS/ "/var/spool/asterisk/monitorDONE/"
<Directory "/var/spool/asterisk/monitorDONE/">
    Options Indexes MultiViews
    AllowOverride None
    Require all granted
    <Files ~ "\.mp3$">
        ForceType application/octet-stream
        Header set Content-Disposition attachment
    </Files>
</Directory>

For AlmaLinux (/etc/httpd/conf.d/recordings.conf):

Alias /RECORDINGS/ "/var/spool/asterisk/monitorDONE/"
<Directory "/var/spool/asterisk/monitorDONE/">
    Options Indexes MultiViews
    AllowOverride None
    Require all granted
</Directory>

Reload Apache:

systemctl reload apache2  # ViciBox
systemctl reload httpd     # AlmaLinux

Fix 2: Fix file permissions

# ViciBox (web server user is wwwrun):
chown -R wwwrun:www /var/spool/asterisk/monitorDONE/
chmod -R 755 /var/spool/asterisk/monitorDONE/

# AlmaLinux (web server user is apache):
chown -R apache:apache /var/spool/asterisk/monitorDONE/
chmod -R 755 /var/spool/asterisk/monitorDONE/

# Or just fix read permissions without changing ownership:
chmod -R o+rx /var/spool/asterisk/monitorDONE/

Fix 3: Configure FTP archive server

In /etc/astguiclient.conf:

VARFTP_host => archive.yourdomain.com
VARFTP_user => recordings
VARFTP_pass => your_ftp_password
VARFTP_port => 21
VARFTP_dir => /RECORDINGS
VARHTTP_path => http://archive.yourdomain.com/RECORDINGS

The VARHTTP_path is critical — this is the base URL that gets stored in the recording_log.location field. It must be accessible from the browser.

On the archive server, configure Apache/Nginx to serve the /RECORDINGS directory.

Fix 4: Fix the database location for existing recordings

If recordings exist on disk but the database has wrong paths:

-- Update recording locations to point to the correct server
UPDATE recording_log
SET location = REPLACE(location, 'http://old-server', 'http://new-server')
WHERE location LIKE '%old-server%';

Fix 5: Fix mixed content (HTTP/HTTPS)

If your admin panel is served over HTTPS but recording links use HTTP:

-- Update all HTTP recording links to HTTPS
UPDATE recording_log
SET location = REPLACE(location, 'http://', 'https://')
WHERE location LIKE 'http://%';

Prevention

• Always configure the Apache RECORDINGS alias during initial setup
• Test recording playback after every server migration or IP change
• Use HTTPS for recording URLs from the start
• Monitor disk space and FTP upload success daily

Verify It Worked

# Test that the RECORDINGS alias works:
curl -sI http://YOUR_SERVER/RECORDINGS/MP3/ | head -5
# Should return "200 OK" (or 200 with directory listing)

# Test a specific recording:
curl -sI "$(mysql -u cron -p1234 -Nse \
  "SELECT location FROM recording_log ORDER BY start_time DESC LIMIT 1;" asterisk)"
# Should return "200 OK"

---

Quick Reference: Port Cheat Sheet

Service	Port	Protocol	Required
SIP signaling	5060	UDP + TCP	Yes
SIP TLS	5061	TCP	For encrypted SIP
IAX2	4569	UDP	If using IAX trunks
RTP media	10000-20000	UDP	Yes
HTTP (web GUI)	80	TCP	Yes
HTTPS	443	TCP	For WebRTC/ViciPhone
WebSocket (WSS)	8089	TCP	For WebRTC/ViciPhone
MySQL	3306	TCP	For DB clusters
AMI (Asterisk Manager)	5038	TCP	Internal only
SSH	22 (or custom)	TCP	Administration
NTP	123	UDP	Time sync

---

Quick Reference: Essential File Paths

File	Purpose
/etc/astguiclient.conf	ViciDial server configuration (server IP, DB creds, keepalive settings)
/etc/asterisk/sip.conf	Asterisk SIP global settings (includes sip-vicidial.conf)
/etc/asterisk/sip-vicidial.conf	ViciDial-managed SIP peers and trunks
/etc/asterisk/pjsip.conf	PJSIP configuration (Asterisk 16+)
/etc/asterisk/extensions.conf	Main dialplan (includes extensions-vicidial.conf)
/etc/asterisk/extensions-vicidial.conf	ViciDial-managed dialplan entries
/etc/asterisk/http.conf	Asterisk HTTP/WebSocket server config
/etc/asterisk/rtp.conf	RTP port range configuration
/etc/asterisk/amd.conf	Answering Machine Detection parameters
/etc/asterisk/manager.conf	Asterisk Manager Interface config
/etc/my.cnf	MySQL/MariaDB configuration
/var/spool/asterisk/monitor/	Active call recordings (raw)
/var/spool/asterisk/monitorDONE/	Processed recordings (ORIG/, MP3/, FTP/)
/var/log/asterisk/messages	Asterisk system log
/var/log/astguiclient/	ViciDial script logs
/usr/share/astguiclient/	ViciDial Perl scripts (all AST_.pl and ADMIN_.pl)

---

Quick Reference: Essential Asterisk CLI Commands

# Check SIP trunk status
asterisk -rx "sip show peers"
asterisk -rx "sip show registry"

# Check active calls
asterisk -rx "core show channels"
asterisk -rx "core show calls"

# Check active conferences
asterisk -rx "meetme list"
asterisk -rx "confbridge list"

# SIP debugging
asterisk -rx "sip set debug on"
asterisk -rx "sip set debug off"

# Reload configurations (non-disruptive to active calls)
asterisk -rx "sip reload"
asterisk -rx "dialplan reload"
asterisk -rx "module reload"

# Check loaded modules
asterisk -rx "module show like websocket"
asterisk -rx "module show like pjsip"

# DAHDI status
asterisk -rx "dahdi show status"

# HTTP/WebSocket status
asterisk -rx "http show status"

---

Quick Reference: Essential SQL Queries

-- Active agents right now
SELECT user, status, campaign_id, last_update_time
FROM vicidial_live_agents ORDER BY status;

-- Calls in the last hour by status
SELECT status, COUNT(*) FROM vicidial_closer_log
WHERE call_date > NOW() - INTERVAL 1 HOUR GROUP BY status;

-- Hopper status per campaign
SELECT campaign_id, COUNT(*) AS leads FROM vicidial_hopper GROUP BY campaign_id;

-- Stuck auto-calls (should be empty)
SELECT * FROM vicidial_auto_calls WHERE call_time < NOW() - INTERVAL 30 MINUTE;

-- Database size by table
SELECT table_name, ROUND((data_length+index_length)/1024/1024,1) AS mb, table_rows
FROM information_schema.tables WHERE table_schema='asterisk'
ORDER BY data_length+index_length DESC LIMIT 15;

-- Agent LAGGED events today
SELECT user, COUNT(*) AS lagged_count
FROM vicidial_agent_log
WHERE event_time > CURDATE() AND sub_status = 'LAGGED'
GROUP BY user ORDER BY lagged_count DESC;

---

This guide covers the 20 most common ViciDial issues based on forum discussions with 30-90+ replies each. For issues not covered here, the primary resources are the ViciDial Forum, the official documentation at vicidial.org/docs, and the ViciDial manager's manual.