RHEL: /data Disappeared After Reboot? Troubleshoot LVM and Multipath Before Touching pvcreate

hosni1982 — Sat, 29 Aug 2026 22:44:42 +0000

A filesystem disappearing after a reboot is one of those incidents where a quick "fix" can make the situation much worse.

Imagine this:

/data is missing
Application cannot start
systemd reports a mount failure
Server may even enter emergency mode

The first reaction might be:

pvcreate
vgcreate
lvcreate
mkfs.xfs

On an existing production LUN, that can be exactly the wrong thing to do.

The problem may not be LVM at all.

Start with the failure layers

In an enterprise RHEL environment, /data can depend on several layers:

/data
  ↓
Filesystem
  ↓
Logical Volume
  ↓
Volume Group
  ↓
Physical Volume
  ↓
Multipath
  ↓
SAN LUN
  ↓
Storage paths

The goal is simple:

Find the first broken layer before changing anything.

1. Is `/data` actually expected to be mounted?

Start with:

findmnt /data
grep -w '/data' /etc/fstab
lsblk -f

Then inspect the boot errors:

journalctl -b -p err..alert --no-pager

You may see something like:

Failed to mount /data
Dependency failed for Local File Systems

That only proves the mount failed.

It does not prove that the filesystem is corrupted.

2. Does the Logical Volume exist?

Check LVM:

lvs -a -o lv_name,vg_name,lv_attr,devices

A healthy configuration might show:

LV       VG       Attr       Devices
lv_data  vg_data  -wi-a----- /dev/mapper/mpatha(0)

If the LV is missing, move down one layer.

Do not recreate it yet.

3. Does the Volume Group exist?

vgs

Expected example:

VG       #PV #LV #SN Attr   VSize    VFree
vg_data    1   1   0 wz--n- 500.00g  20.00g

If vg_data is missing, resist the temptation to run vgcreate.

Check the PV first.

4. Is the Physical Volume visible?

pvs -o pv_name,vg_name,pv_size,pv_free

Expected:

PV                  VG       PSize    PFree
/dev/mapper/mpatha  vg_data  500.00g  20.00g

If /dev/mapper/mpatha is missing, the problem is probably below LVM.

Now the investigation changes direction.

5. Check Multipath

multipath -ll

Example of a healthy map:

mpatha (3600508b400105df70000e00000ac0000) dm-3
size=500G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw

|-+- policy='service-time 0' prio=50 status=active
| `- 2:0:0:1 sdb 8:16 active ready running

`-+- policy='service-time 0' prio=10 status=enabled
  `- 3:0:0:1 sdc 8:32 active ready running

You can inspect the individual paths with:

multipathd show paths

and:

lsscsi

If the expected WWID is absent completely, recreating the VG will not fix the root cause.

The SAN LUN may simply not be visible to the server.

A typical failure chain

Suppose your checks give this result:

findmnt /data
    → not mounted

lvs
    → lv_data missing

vgs
    → vg_data missing

pvs
    → expected PV missing

multipath -ll
    → expected WWID missing

Now the picture is very different:

SAN LUN not visible
        ↓
Multipath map missing
        ↓
PV missing
        ↓
VG missing
        ↓
LV missing
        ↓
Filesystem unavailable
        ↓
/data not mounted

The missing filesystem was only the visible symptom.

The real failure occurred much lower in the stack.

Commands I would avoid at this stage

Unless you have proven that the storage is new and intentionally being initialized, avoid commands such as:

pvcreate /dev/mapper/mpatha
vgcreate vg_data /dev/mapper/mpatha
lvcreate ...
mkfs.xfs ...

These are creation commands, not troubleshooting commands.

On an existing production disk, they may overwrite metadata you are trying to recover.

Evidence to collect before escalation

Before calling the storage, SAN, VMware or infrastructure team, collect evidence:

date
hostnamectl
uname -r
lsblk -f
findmnt
pvs
vgs
lvs -a -o +devices
multipath -ll
multipathd show paths
journalctl -b -p err..alert --no-pager

In complex cases, and if permitted by your environment:

sos report

This gives the next support level something concrete to work with.

The troubleshooting habit that matters

When /data disappears, don't immediately ask:

Which command can recreate /data?

Ask:

Which command proves why /data disappeared?

That small change in mindset can prevent a recoverable infrastructure incident from becoming a data-loss incident.

Free Linux troubleshooting resources

I maintain a small public project called Linux Ops Rescue Kit with free RHEL troubleshooting examples and read-only diagnostic tools.

The GitHub repository currently includes:

High load but low CPU troubleshooting
LVM / Multipath missing filesystem troubleshooting
A read-only Linux snapshot script
An evidence-first troubleshooting methodology

GitHub:

https://github.com/hosni1982/linux-ops-rescue-kit

The free release is also available here:

https://github.com/hosni1982/linux-ops-rescue-kit/releases/tag/v1.0-free

For engineers who need a larger on-call runbook, the complete Linux Ops Rescue Kit contains 30 production incident workflows, verification, rollback and Stop & Escalate guidance.

If you work on RHEL production systems, I'd be interested to know:

What is the most dangerous "quick fix" you've seen during a storage incident?

DEV Community: hosni1982