DEV Community

Cover image for Arch Linux AUR Dependency Hell: Debugging the ICU 76 78 Migration
hopsayer
hopsayer

Posted on

Arch Linux AUR Dependency Hell: Debugging the ICU 76 78 Migration

#archlinux #help #linux

Fixing ICU Library Dependency Hell on Arch Linux: A Deep Dive into Shared Library Mismatches

The Problem

I recently encountered a frustrating issue when trying to launch Notepadqq on my Arch-based system (EndeavourOS):

$ notepadqq 
/usr/bin/../lib/notepadqq/notepadqq-bin: error while loading shared libraries: 
libicui18n.so.76: cannot open shared object file: No such file or directory
Enter fullscreen mode Exit fullscreen mode

This is a classic shared library dependency problem that's particularly common on rolling-release distributions like Arch Linux.

Understanding the Root Cause

What is ICU?

ICU (International Components for Unicode) is a mature, widely used set of C/C++ libraries providing Unicode and globalization support. Many applications depend on it, including Qt's WebEngine component.

The Rolling Release Challenge

The issue stemmed from a critical version mismatch between official Arch repositories and AUR packages:

What happened:

  • Arch upstream had moved forward to ICU 78 in official repositories
  • Notepadqq (an AUR package) was still compiled against ICU 76
  • Qt5WebEngine had been removed from official repos but remained on my system, still linked to ICU 76

The core problem: Notepadqq hasn't been actively maintained since 2018. As an AUR package, it doesn't get automatically rebuilt when Arch updates system libraries like ICU. This created a desynchronization between:

  • Official repository packages (ICU 78)
  • Outdated AUR binaries (expecting ICU 76)
  • Orphaned repository packages (Qt5WebEngine with ICU 76 linkage)

This cascading dependency mismatch is a common pitfall when mixing official packages with unmaintained AUR software.

Diagnostic Steps

1. Check Current ICU Version 

$ ls /usr/lib/libicui18n.so*
/usr/lib/libicui18n.so
/usr/lib/libicui18n.so.78
/usr/lib/libicui18n.so.78.1
Enter fullscreen mode Exit fullscreen mode

This confirmed I had ICU 78, while the application expected version 76.

2. Identify Package Origin 

$ pacman -Ss notepadqq
$                         # Returns empty line
$ yay notepadqq
$ 1 aur/notepadqq 2.0.0beta-3 (+1 0.32) (Installed)
Enter fullscreen mode Exit fullscreen mode

This confirmed notepadqq was installed, but being an AUR package (not from official repos), it wouldn't automatically rebuild when system libraries like ICU were updated.

3. Attempt to Rebuild

When I tried rebuilding Notepadqq from AUR:

$ yay -S notepadqq --rebuild
Enter fullscreen mode Exit fullscreen mode

I hit a deeper issue:

/usr/bin/ld: /usr/lib/libQt5WebEngineCore.so: undefined reference to 
`icu_76::Normalizer::normalize(...)`
/usr/bin/ld: /usr/lib/libQt5WebEngineCore.so: undefined reference to 
`ucnv_open_76'
Enter fullscreen mode Exit fullscreen mode

4. Discover the Real Culprit 

$ pacman -Qo /usr/lib/libQt5WebEngineCore.so
/usr/lib/libQt5WebEngineCore.so is owned by qt5-webengine 5.15.19-4
Enter fullscreen mode Exit fullscreen mode

The problem wasn't just Notepadqq—Qt5WebEngine itself was compiled against the old ICU version.

5. Repository Discovery

When trying to update qt5-webengine:

$ sudo pacman -Syu qt5-webengine
error: target not found: qt5-webengine
Enter fullscreen mode Exit fullscreen mode

Critical finding: qt5-webengine had been removed from official Arch repositories but remained installed on my system as an orphaned package.

The Solution

Step 1: Remove Conflicting Packages 

$ sudo pacman -Rns qt5-webengine notepadqq
Enter fullscreen mode Exit fullscreen mode

This removed:

  • qt5-webengine (159.19 MiB)
  • notepadqq (5.89 MiB)
  • Related dependencies (qt5-location, qt5-webchannel, qt5-websockets)

Total freed: 174.88 MiB

Step 2: Rebuild Qt5WebEngine from AUR 

$ yay -S qt5-webengine
Enter fullscreen mode Exit fullscreen mode

This compiled qt5-webengine from source, linking it against the current ICU 78.

Step 3: Rebuild Notepadqq 

$ yay -S notepadqq
Enter fullscreen mode Exit fullscreen mode

With qt5-webengine now properly linked to ICU 78, Notepadqq compiled successfully.

Step 4: Verification 

$ ldd /usr/bin/notepadqq | grep icu
        libicui18n.so.78 => /usr/lib/libicui18n.so.78
        libicuuc.so.78 => /usr/lib/libicuuc.so.78
Enter fullscreen mode Exit fullscreen mode

Success! The application now links against the correct ICU version.

Key Lessons Learned

1. AUR Packages Don't Auto-Update with System Libraries

Critical insight: When Arch updates a core library like ICU, official packages are rebuilt automatically. AUR packages are not—they must be manually rebuilt by users or maintainers.

Notepadqq was last updated in 2018, meaning:

  • It never received rebuilds for ICU 76, 77, or 78
  • Each ICU update broke the package until manually rebuilt
  • This is a fundamental difference between official repos and AUR

2. Partial Upgrades Are Dangerous

Even though I was updating my system 2-5 times daily, I hadn't rebooted in 7 days. However, rebooting wasn't the issue — the problem was binary incompatibility on disk.

2. Rolling Release Requires Package Consistency

Arch's rolling release model means:

  • Libraries like ICU update frequently
  • All dependent packages must be rebuilt or updated in sync
  • Orphaned packages from removed repositories can cause hidden issues

3. ABI Compatibility Matters

The ICU version number in the shared library name (libicui18n.so.76 vs libicui18n.so.78) indicates an ABI break. You cannot simply symlink one version to another without risking crashes or undefined behavior.

Recognizing an ABI Break

How did I know this wasn't just a missing symlink issue?

1. Version in library filename:

  • libicui18n.so.76 vs libicui18n.so.78
  • The .76 and .78 are SONAME major versions
  • Different major versions = intentional ABI incompatibility

2. Versioned symbols in linker errors:

  • icu_76::Normalizer::normalize ← version baked into symbol name
  • ucnv_open_76 ← function name includes version
  • The library exposes icu_78:: symbols, but the binary needs icu_76::
  • No amount of symlinking can resolve this mismatch

3. Why symlinking would fail:
Even if we created ln -s libicui18n.so.78 libicui18n.so.76:

  • The binary would load, but call icu_76:: functions
  • The library only has icu_78:: functions
  • Result: immediate crash or undefined behavior

4. Dependency Chains Run Deep

What appeared to be a simple Notepadqq issue actually required:

  1. Diagnosing the ICU mismatch
  2. Discovering the Qt5WebEngine dependency
  3. Identifying the orphaned repository package
  4. Rebuilding the entire dependency chain

What NOT to Do

❌ Creating Symlinks 

# DON'T DO THIS
$ sudo ln -s /usr/lib/libicui18n.so.78 /usr/lib/libicui18n.so.76
Enter fullscreen mode Exit fullscreen mode

While this might allow the program to start, it can cause:

  • Runtime crashes
  • Undefined behavior
  • Data corruption
  • Difficult-to-debug issues

❌ Ignoring Update Errors

Regularly running pacman -Syu is good, but you must also:

  • Check for orphaned packages: pacman -Qtdq
  • Verify AUR packages after system library updates
  • Review failed transaction logs

Prevention Strategies

1. Regular Orphan Package Cleanup 

$ pacman -Qtdq | pacman -Rns -
Enter fullscreen mode Exit fullscreen mode

2. Monitor AUR Package Build Failures

When a system library updates, rebuild affected AUR packages immediately:

$ yay -S --rebuild <package-name>
Enter fullscreen mode Exit fullscreen mode

Important: Unmaintained AUR packages (like Notepadqq, last updated 2018) will always need manual rebuilds after system library updates. Consider:

  • Finding actively maintained alternatives
  • Adopting the AUR package yourself
  • Using AppImage/Flatpak versions that bundle dependencies

3. Use Binary Package Alternatives

For stability-critical applications, consider -bin packages from AUR that bundle their dependencies, though this increases disk usage.

4. Check Library Dependencies

After major library updates:

$ ldd /usr/bin/<application> | grep "not found"
Enter fullscreen mode Exit fullscreen mode

Conclusion

This issue perfectly demonstrates why Arch Linux is considered an "advanced" distribution. The rolling release model provides cutting-edge packages but requires users to understand:

  • Shared library versioning (ABI compatibility)
  • Dependency chains
  • The relationship between official repositories and AUR
  • When to rebuild vs. reinstall packages

The solution wasn't a quick fix but rather a systematic diagnosis and rebuild of the dependency chain. While frustrating in the moment, issues like this deepen understanding of how Linux systems work at a fundamental level.

Have you encountered similar dependency issues on Arch? What's your approach to handling library version mismatches? Share your experiences in the comments!

System Info:

  • Distribution: EndeavourOS (Arch-based)
  • Package Manager: pacman + yay
  • Affected Versions: ICU 76 → 78, Qt5WebEngine 5.15.19-4

Top comments (0)