During a routine security audit, the team identified an issue on the Nautilus App Server. Some malicious content was identified within the website code. After digging into the issue they found that there might be more infected files. Before doing a cleanup they would like to find all similar files and copy them to a safe location for further investigation. Accomplish the task as per the following requirements:
a. On App Server 1 at location /var/www/html/media find out all files (not directories) having .php extension.
b. Copy all those files along with their parent directory structure to location /media on same server.
c. Please make sure not to copy the entire /var/www/html/media directory content.
Introduction
In the world of system administration, there are times when you need to locate specific files across a directory structure and copy them to a safe location while preserving their original directory hierarchy. This is particularly common during security audits, malware investigations, or when implementing file-based backups.
In this comprehensive guide, I will walk through the process of finding all PHP files in a directory and copying them with their directory structure intact without copying the entire directory content. This tutorial is based on a real-world scenario where malicious content was identified in website code and needed to be isolated for investigation.
Understanding the Problem
The Scenario
During a routine security audit, a team identified malicious content in website code on a Nautilus App Server. The investigation revealed that there might be more infected files. Before performing cleanup, the team needed to find all similar files, specifically PHP files, copy them to a safe location for investigation, preserve the original directory structure, and avoid copying the entire directory content.
The Challenge
Source: /var/www/html/media
Target: /media
File Type: .php files only
Requirement: Preserve directory structure
Why Preserve Directory Structure
Preserving directory structure is important for several reasons. It provides context about where files came from, which helps understand the attack vector. The directory structure reveals file organization and relationships between files. If restoration is needed, files can be placed back to their exact original locations. Understanding the file hierarchy also helps identify patterns during investigation.
Prerequisites
Before beginning, ensure you have SSH access to the target server with root or sudo privileges. You should have understanding of Linux file permissions and basic command-line knowledge.
For this tutorial, the server details are:
- Server: App Server 1 (stapp01)
- User: tony
- Password: Ir0nM@n
- Source Directory: /var/www/html/media
- Destination Directory: /media
- File Type: .php files only
Understanding the Find Command
Basic Find Syntax
find [path] [options] [expression]
Components for This Task
find /var/www/html/media -type f -name "*.php" -print0
The command breaks down as follows. /var/www/html/media is the starting directory to search. -type f ensures only files are found, not directories. -name "*.php" matches files ending with the .php extension. -print0 prints results with a null character, which properly handles filenames containing spaces.
Why Use -print0
When filenames contain spaces, the default output can break commands that process the results. The -print0 option solves this by using a null character as a separator instead of a newline. This ensures proper handling of all filenames regardless of the characters they contain.
The Challenge of Preserving Directory Structure
When copying files found by the find command, the default behavior is to flatten the structure, placing all files directly into the destination directory. The goal is to preserve the full directory hierarchy, which requires specific techniques.
Method 1: Using CPIO
What is CPIO
CPIO, which stands for Copy In/Out, is a file archiving utility that is perfect for this task because it can copy files with their directory structure intact.
The Command
find /var/www/html/media -type f -name "*.php" -print0 | cpio -pdmv0 /media
Command Breakdown
The find portion locates all PHP files and outputs them with null termination. The pipe sends this list to cpio. CPIO operates in pass-through mode using the -p option. The -d option creates directories as needed. The -m option preserves modification times. The -v option provides verbose output showing what is being copied. The -0 option reads null-terminated filenames.
Example Output
[root@stapp01 ~]# find /var/www/html/media -type f -name "*.php" -print0 | cpio -pdmv0 /media
/var/www/html/media/index.php
/var/www/html/media/wp-mail.php
/var/www/html/media/wp-links-opml.php
/var/www/html/media/wp-load.php
/var/www/html/media/xmlrpc.php
/var/www/html/media/wp-login.php
... (list continues)
1288 blocks
Pros and Cons
The CPIO method preserves directory structure and handles filenames with spaces. It is fast for large numbers of files and preserves file permissions. However, CPIO may not be installed by default on all systems, and some administrators are less familiar with this utility compared to more common tools.
Method 2: Using Tar
The Command
find /var/www/html/media -type f -name "*.php" -print0 | tar -cf - --null -T - | tar -xf - -C /media
How It Works
The find command locates all PHP files. The output is piped to tar, which creates an archive to stdout. The --null -T - options tell tar to read null-terminated filenames from stdin. The second tar command extracts the archive from stdin to the destination directory.
Step-by-Step Breakdown
The first part finds all PHP files and creates a tar archive, sending it to stdout. The second part extracts the archive from stdin to the media directory.
Example Output
[root@stapp01 ~]# find /var/www/html/media -type f -name "*.php" -print0 | tar -cf - --null -T - | tar -xf - -C /media
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
Pros and Cons
The tar method is beneficial because tar is almost always installed on Linux systems. It preserves directory structure and handles filenames with spaces effectively. On the downside, the syntax is slightly more complex than other methods, and a temporary archive is created in memory.
Method 3: Using CP with --parents
The Command
find /var/www/html/media -type f -name "*.php" -exec cp --parents {} /media \;
How It Works
The find command locates all PHP files. The -exec option executes the cp command for each file found. The --parents option preserves the directory structure. The {} placeholder is replaced with the current file path, and the \; ends the -exec command.
Alternative Using xargs
For better performance with many files, xargs can be used:
find /var/www/html/media -type f -name "*.php" -print0 | xargs -0 cp --parents -t /media
Example Output
[root@stapp01 ~]# find /var/www/html/media -type f -name "*.php" -exec cp --parents {} /media \;
[root@stapp01 ~]# ls -la /media/var/www/html/media/
total 216
drwxr-xr-x 5 root root 4096 Sep 5 18:03 .
drwxr-xr-x 3 root root 4096 Sep 5 18:03 ..
-rw-r--r-- 1 root root 405 Sep 5 18:03 index.php
-rw-r--r-- 1 root root 7349 Sep 5 18:03 wp-activate.php
...
Pros and Cons
The cp method is simple and readable, and cp is always available on Linux systems. The directory structure is preserved, and the command is easy to understand. However, executing cp for each file individually can be slower for thousands of files compared to methods that process files in batches.
Method 4: Using Rsync
The Command
find /var/www/html/media -type f -name "*.php" -print0 | rsync -av --files-from=- --from0 / /media/
How It Works
The find command locates all PHP files. Rsync copies files from the list provided. The --files-from=- option tells rsync to read the file list from stdin. The --from0 option uses null-terminated input. The / is the source root, and /media/ is the destination.
Example
[root@stapp01 ~]# find /var/www/html/media -type f -name "*.php" -print0 | rsync -av --files-from=- --from0 / /media/
building file list ... done
var/www/html/media/index.php
var/www/html/media/wp-mail.php
...
sent 1.23M bytes received 0.00 bytes 123.00K bytes/sec
total size is 1.20M speedup is 0.98
Pros and Cons
Rsync is very efficient for large numbers of files and shows progress and statistics. It can handle incremental copies and preserves permissions and timestamps. However, rsync may not be installed by default, and the syntax is more complex than simpler methods.
Method 5: Using a Bash Script
The Script
#!/bin/bash
SOURCE_DIR="/var/www/html/media"
DEST_DIR="/media"
FILE_PATTERN="*.php"
mkdir -p "$DEST_DIR"
echo "Finding and copying .php files..."
find "$SOURCE_DIR" -type f -name "$FILE_PATTERN" | while read -r file; do
rel_path="${file#$SOURCE_DIR/}"
mkdir -p "$DEST_DIR/$(dirname "$rel_path")"
cp "$file" "$DEST_DIR/$rel_path"
echo "Copied: $rel_path"
done
SOURCE_COUNT=$(find "$SOURCE_DIR" -type f -name "$FILE_PATTERN" | wc -l)
DEST_COUNT=$(find "$DEST_DIR$SOURCE_DIR" -type f -name "$FILE_PATTERN" 2>/dev/null | wc -l)
echo ""
echo "Summary"
echo "Source .php files: $SOURCE_COUNT"
echo "Copied .php files: $DEST_COUNT"
if [ $SOURCE_COUNT -eq $DEST_COUNT ]; then
echo "All files copied successfully"
else
echo "File count mismatch"
fi
Running the Script
Save the script, make it executable, and run it.
chmod +x copy_php_files.sh
./copy_php_files.sh
Pros and Cons
A bash script is fully customizable and shows progress. It is easy to modify for different requirements and can handle edge cases. The main drawback is that it is slower for large numbers of files compared to batch-processing methods and requires scripting knowledge.
Verification and Validation
Key Verification Commands
After copying files, verify the operation with these commands. Count PHP files in the source directory to establish a baseline. Count PHP files in the destination directory and compare the numbers. They should match exactly.
Verify that the directory structure is preserved by listing the contents recursively. Check that no non-PHP files were copied to the destination. Compare file sizes between source and destination to ensure file integrity.
View a sample of the copied files to confirm the operation worked as expected.
Complete Verification Script
#!/bin/bash
SOURCE="/var/www/html/media"
DEST="/media"
PATTERN="*.php"
SOURCE_COUNT=$(find "$SOURCE" -type f -name "$PATTERN" | wc -l)
DEST_COUNT=$(find "$DEST$SOURCE" -type f -name "$PATTERN" 2>/dev/null | wc -l)
echo "Source files: $SOURCE_COUNT"
echo "Destination files: $DEST_COUNT"
if [ $SOURCE_COUNT -eq $DEST_COUNT ]; then
echo "Counts match"
else
echo "Count mismatch"
fi
echo "Top-level directories in destination:"
ls -la "$DEST$SOURCE" 2>/dev/null | head -10
echo "Sample Files:"
find "$DEST$SOURCE" -type f -name "$PATTERN" 2>/dev/null | head -10
NON_PHP=$(find "$DEST" -type f ! -name "$PATTERN" 2>/dev/null | wc -l)
echo "Non-PHP files in destination: $NON_PHP"
if [ $NON_PHP -eq 0 ]; then
echo "Only .php files were copied"
fi
Troubleshooting Common Issues
cpio Command Not Found
When cpio is not installed on the system, the command will fail. The solution is to install cpio using the package manager or use an alternative method such as tar or cp with --parents.
No Such File or Directory
When the source directory does not exist, the find command will return an error. Check the directory path and adjust if necessary by searching for the correct location.
Permission Denied
Insufficient permissions can prevent reading source files or writing to the destination. Use sudo or become root to run the commands with elevated privileges.
tar Removing Leading Slash Warning
This warning is normal behavior when creating tar archives with absolute paths. It can be safely ignored. To suppress it, use the -C option to change directory before archiving.
Filenames with Spaces
When filenames contain spaces, they are not handled correctly without proper null termination. Always use -print0 with find and the corresponding -0 or --null options with the receiving command.
Destination Already Has Files
When the destination directory already contains files, decide whether to clear it first, copy without overwriting, or create a backup of existing files.
Best Practices
Test First
Before running the full operation, test with find alone to see which files will be matched. Then proceed with the full command.
Use Dry Run Options
Many tools offer dry run options that show what would happen without making changes. Rsync has -n for dry run, and tar can list files without extracting.
Create a Backup
Before making significant changes, create a backup of the destination directory to protect against data loss.
Use Absolute Paths
Using absolute paths in scripts and commands provides clarity and prevents errors from relative path resolution.
Log Your Actions
Redirect the output of the operation to a log file for future reference and auditing. This provides a record of what was copied and any errors encountered.
Verify File Integrity
After copying files, verify the integrity by comparing file counts, sizes, and checksums between source and destination.
Preserve Metadata
When metadata such as timestamps and permissions are important, use methods that preserve these attributes. Both CPIO and Rsync preserve metadata when used with the appropriate options.
Conclusion
What Has Been Accomplished
Finding specific files across a directory hierarchy and copying them to a safe location while preserving the directory structure is a common system administration task. This is particularly important during security investigations when suspicious files need to be isolated for analysis.
The task involves several key components. The find command locates all files matching specific criteria such as file type and name pattern. The copying method preserves the full directory structure without flattening it. Verification ensures that all files were copied correctly and no unintended files were included.
Method Comparison
The CPIO method is recommended when available because it is fast, preserves metadata, and handles filenames with spaces properly. The only drawback is that CPIO may not be installed by default.
The tar method is a great fallback when CPIO is not available. It is widely available on all Linux systems and handles filenames with spaces correctly. The syntax is more complex than other methods.
The cp with --parents method is the simplest option and is always available. It is easy to understand and works well for moderate numbers of files. It can be slower for very large numbers of files.
The rsync method is the most efficient for large operations and provides progress feedback. It is ideal when rsync is already installed and when incremental copying is needed.
The bash script method offers complete customization and is useful for complex requirements or when integrating with other processes.
Key Takeaways
Using -type f with find ensures only files are found, not directories. The -print0 option properly handles filenames with spaces. CPIO in pass-through mode is the most efficient method when available. Tar is a reliable fallback when CPIO is not available. The cp command with --parents is simple and always available.
Always verify file counts match after copying to ensure completeness. Preserve directory structure for context and investigation purposes. Use absolute paths for clarity and reliability. Log operations for auditing and troubleshooting.
Complete Solution Summary
The following methods can be used to find and copy PHP files while preserving directory structure.
For CPIO:
find /var/www/html/media -type f -name "*.php" -print0 | cpio -pdmv0 /media
For Tar:
find /var/www/html/media -type f -name "*.php" -print0 | tar -cf - --null -T - | tar -xf - -C /media
For CP with Parents:
find /var/www/html/media -type f -name "*.php" -exec cp --parents {} /media \;
For verification:
echo "Source: $(find /var/www/html/media -type f -name '*.php' | wc -l)"
echo "Destination: $(find /media/var/www/html/media -type f -name '*.php' | wc -l)"
Quick Reference Card
Find PHP files:
find /path -type f -name "*.php"
Count PHP files:
find /path -type f -name "*.php" | wc -l
Copy with CPIO:
find ... -print0 | cpio -pdmv0 /dest
Copy with Tar:
find ... -print0 | tar -cf - --null -T - | tar -xf - -C /dest
Copy with CP:
find ... -exec cp --parents {} /dest \;
Verify counts:
Compare wc -l outputs
Check directory structure:
ls -laR /dest/path
Additional Resources
Manual pages are available for all the commands used. The find, cpio, tar, cp, and rsync manuals provide detailed documentation on all available options.
Related topics include Linux file permissions, shell scripting, security auditing, and backup strategies. Understanding these areas enhances overall system administration capabilities.
Automation tools such as Ansible can also be used for similar operations in infrastructure-as-code environments.
This guide was created based on a real-world implementation of finding and copying PHP files for security investigation on a Nautilus App Server in the Stratos Datacenter.
Top comments (0)