DEV Community: Bohdan Harabadzhyu

Working with FtpWebRequest in C# using WhisperFTPApp

Bohdan Harabadzhyu — Wed, 05 Feb 2025 04:59:27 +0000

Introduction

When working with FTP (File Transfer Protocol) servers in .NET, the FtpWebRequest class is one of the most commonly used tools. FtpWebRequest is a class designed specifically for handling FTP operations in .NET. It inherits from the WebRequest class, which is a base class for making requests to various types of servers (HTTP, FTP, etc.).
In this article, we’ll explore how to use FtpWebRequest by walking through the implementation of my FTP Client project, which is available on GitHub and SourceForge. If you’d like to see the project in action, I’ve also created a video overview on YouTube.
This article will focus specifically on the FTP-related functionality of the application, using FtpWebRequest from the System.Net namespace. We won’t be diving into the Avalonia UI framework or the MVVM pattern, as the goal here is to concentrate on the core FTP operations and how they are implemented in the project.

Project Structure

The WhisperFTPApp project is structured to separate concerns. Here’s a breakdown of the key components:
IFtpService Interface

This interface defines the contract for all FTP operations. It abstracts the implementation details, making it easier to swap out the FTP service if needed.

public interface IFtpService
{
    Task<bool> ConnectAsync(FtpConfiguration configuration);
    Task DisconnectAsync();
    Task<IEnumerable<FileSystemItem>> ListDirectoryAsync(FtpConfiguration configuration, string path = "/");
    Task UploadFileAsync(FtpConfiguration configuration, string localPath, string remotePath, IProgress<double> progress);
    Task DownloadFileAsync(FtpConfiguration configuration, string remotePath, string localPath, IProgress<double> progress);
    Task DeleteFileAsync(FtpConfiguration configuration, string remotePath);
    Task DeleteDirectoryAsync(FtpConfiguration configuration, string path);
}

FtpConfiguration Class

This class stores the configuration settings required to connect to an FTP server.

public class FtpConfiguration
{
    public string FtpAddress { get; }
    public string Username { get; }
    public string Password { get; }
    public int Port { get; }
    public int Timeout { get; }

    public FtpConfiguration(string ftpAddress, string username, string password, int port, int timeout)
    {
        FtpAddress = ftpAddress;
        Username = username;
        Password = password;
        Port = port;
        Timeout = timeout;
    }
}

FtpService Class

This class implements the IFtpService interface and provides the actual implementation of the FTP operations using FtpWebRequest. In this article, I’ll be focusing on the ConnectAsync method, breaking down how it interacts with FtpWebRequest to establish a connection.

public class FtpService : IFtpService
{
    private FtpWebRequest _currentRequest;
    private const int MaxRetries = 3;
    private const int BaseDelay = 1000;

    public async Task<bool> ConnectAsync(FtpConfiguration configuration)
    {
        StaticFileLogger.LogInformation($"Attempting to connect to {configuration.FtpAddress}");
        for (int attempt = 1; attempt <= MaxRetries; attempt++)
        {
            try
            {
                StaticFileLogger.LogInformation($"Connection attempt {attempt} of {MaxRetries}");
                var connectTask = SendRequest(configuration);
                var timeoutTask = Task.Delay(configuration.Timeout);
                var completedTask = await Task.WhenAny(connectTask, timeoutTask);

                if (completedTask == connectTask)
                {
                    return await connectTask;
                }

                StaticFileLogger.LogError($"Connection attempt {attempt} timed out");
                if (attempt < MaxRetries)
                {
                    int delay = BaseDelay * (int)Math.Pow(2, attempt - 1);
                    StaticFileLogger.LogInformation($"Waiting {delay}ms before retry");
                    await Task.Delay(delay);
                }
            }
            catch (WebException ex)
            {
                StaticFileLogger.LogError($"Connection attempt {attempt} failed: {ex.Message}");
                if (attempt == MaxRetries)
                {
                    return IsAuthenticationError(ex);
                }

                int delay = BaseDelay * (int)Math.Pow(2, attempt - 1);
                await Task.Delay(delay);
            }
        }

        StaticFileLogger.LogError("All connection attempts failed");
        return false;
    }

    private async Task<bool> SendRequest(FtpConfiguration configuration)
    {
        var uriBuilder = new UriBuilder(configuration.FtpAddress)
        {
            Port = configuration.Port
        };

        _currentRequest = (FtpWebRequest)WebRequest.Create(uriBuilder.Uri);
        _currentRequest.Method = WebRequestMethods.Ftp.ListDirectory;
        _currentRequest.Credentials = new NetworkCredential(configuration.Username, configuration.Password);
        _currentRequest.Timeout = configuration.Timeout;
        _currentRequest.KeepAlive = false;

        using (FtpWebResponse response = (FtpWebResponse)await _currentRequest.GetResponseAsync())
        {
            return response.StatusCode == FtpStatusCode.OpeningData ||
                   response.StatusCode == FtpStatusCode.AccountNeeded;
        }
    }

// Other methods omitted for brevity...
}

Understanding the Connection Process

The ConnectAsync method is a critical part of the application, as it ensures that the client can reliably connect to the server before performing any operations. Let’s break down the key components and understand their purpose:

_currentRequest This field holds the current FtpWebRequest object. It’s used to manage the connection and send requests to the FTP server.
MaxRetries This defines how many times the application will try to connect to the server before throwing in the towel. Sometimes, connections fail due to temporary issues like network hiccups or server overload. By setting a maximum number of retries, we give the application a chance to recover from these hiccups without giving up too quickly.
BaseDelay This constant defines the initial wait time (in milliseconds) between retry attempts. But here’s the clever part: the delay increases exponentially with each retry. For example, if the first retry happens after 1 second, the next might wait 2 seconds, then 4 seconds, and so on. This "backoff" strategy prevents the application from bombarding the server with too many requests in a short time, which could make things worse.

Then we go to our method and see that it starts with logging. This is a very useful approach for debugging and monitoring the application’s behavior.
The method uses a for loop to attempt the connection up to MaxRetries times. This ensures that temporary issues (like network glitches) don’t immediately cause the connection to fail.

Connection Attempt and Handling the Result

...

StaticFileLogger.LogInformation($"Connection attempt {attempt} of {MaxRetries}");
var connectTask = SendRequest(configuration);
var timeoutTask = Task.Delay(configuration.Timeout);
var completedTask = await Task.WhenAny(connectTask, timeoutTask);

if (completedTask == connectTask)
  {
     return await connectTask;
  }

...

The SendRequest method is called to create and send the FtpWebRequest. It takes the configuration details (like the server address, port, username, and password) and uses them to build the request. This includes setting the request type, adding the credentials, and defining how long the request should wait before timing out. Once everything is set up, it sends the request and waits for the server’s response..
A Task.Delay is used to create a timeout task. If the connection takes too long (longer than the specified timeout), the timer will "ring," and we’ll know something’s wrong. The Task.WhenAny method waits for either the connection task or the timeout task to complete. If the timeout task completes first, the connection attempt is considered failed.
If the connection task completes first, the method returns the result of the connection attempt (either true or false).

Handling Timeouts and Handling Exceptions

...

StaticFileLogger.LogError($"Connection attempt {attempt} timed out");
  if (attempt < MaxRetries)
    {
      int delay = BaseDelay * (int)Math.Pow(2, attempt - 1);
      StaticFileLogger.LogInformation($"Waiting {delay}ms before retry");
      await Task.Delay(delay);
    }
  }
     catch (WebException ex)
        {
         StaticFileLogger.LogError($"Connection attempt {attempt} failed: {ex.Message}");
          if (attempt == MaxRetries)
            {
              eturn IsAuthenticationError(ex);
             }
             int delay = BaseDelay * (int)Math.Pow(2, attempt - 1);
            await Task.Delay(delay);
         }
   }

...

If the connection attempt times out, the method logs the error and calculates a delay before the next retry. The delay increases exponentially with each attempt to avoid overwhelming the server.

The SendRequest Method

The SendRequest method is responsible for creating and sending the FtpWebRequest. Here’s how it works:


private async Task<bool> SendRequest(FtpConfiguration configuration)
{
    var uriBuilder = new UriBuilder(configuration.FtpAddress)
    {
        Port = configuration.Port
    };

    _currentRequest = (FtpWebRequest)WebRequest.Create(uriBuilder.Uri);
    _currentRequest.Method = WebRequestMethods.Ftp.ListDirectory;
    _currentRequest.Credentials = new NetworkCredential(configuration.Username, configuration.Password);
    _currentRequest.Timeout = configuration.Timeout;
    _currentRequest.KeepAlive = false;

...
}

URI Construction: The method constructs the FTP URI using the UriBuilder class, which ensures that the address and port are correctly formatted.

Setting Properties: Credentials, timeout, and other configurations are applied. _currentRequest.Method: Set to ListDirectory to test the connection.
_currentRequest.Credentials: Set using the provided username and password. _currentRequest.Timeout: Set to the specified timeout value. _currentRequest.KeepAlive: Set to false to ensure that the connection is closed after the request is completed.

Handling the Response

...
     using (FtpWebResponse response = (FtpWebResponse)await _currentRequest.GetResponseAsync())

       {
            return response.StatusCode == FtpStatusCode.OpeningData ||
                   response.StatusCode == FtpStatusCode.AccountNeeded;
       }
     }
...

It uses a using statement to ensure that FtpWebResponse is properly disposed of, managing network resources efficiently. The method checks for status codes like OpeningData (150) and AccountNeeded (532), signaling a ready or responsive server. These codes are essential for identifying a successful connection, even if additional authentication is needed. Integrated into ConnectAsync, this method tests the server's readiness by attempting a directory listing. By differentiating successful connections from failures, the method enhances reliability. As a result, this strategy makes the FTP connection process more dependable.

Using IFtpService in the MainWindowViewModel

The MainWindowViewModel class in WhisperFTPApp manages the interaction between the main window and IFtpService, processing FTP requests. The ConnectToFtpAsync method takes the server address, credentials, and port, validates them, and attempts to establish a connection. Upon success, it updates the UI and loads the list of files and directories from the server.

Initial Setup and Input Validation

public class MainWindowViewModel : ViewModelBase
{
...
    //Other properties and fields omitted for brevity

    private async Task ConnectToFtpAsync()
        {
            IsTransferring = true;
            StaticFileLogger.LogInformation($"Attempting to connect to {FtpAddress}");
            try
                {
                    StatusMessage = "Connecting to FTP server...";
                    FtpItems?.Clear();
                    UpdateRemoteStats();

                    if (!int.TryParse(Port, out int portNumber))
                    {
                        StatusMessage = "Invalid port number";
                        return;
                    }
                    StatusMessage = "Invalid port number";
                     return;
                }

                if (!FtpAddress.StartsWith("ftp://", StringComparison.OrdinalIgnoreCase))
                {
                    FtpAddress = $"ftp://{FtpAddress}";
                }
            FtpAddress = $"ftp://{FtpAddress}";
        }
...

Before attempting to connect, the method sets IsTransferring = true, indicating that an operation is in progress. A log entry is created: "Attempting to connect to [FtpAddress]".
The UI updates to reflect the connection attempt, and the list of remote files is cleared. The method then validates the user’s input. If the port number is invalid, an error message is displayed, and the process stops. Similarly, if the FTP address is missing the ftp:// prefix, it is corrected before proceeding. Once these checks are complete, the connection attempt begins.

Creating FTP Configuration and Attempting Connection

...
        var uri = new Uri(FtpAddress);
        var baseAddress = $"{uri.Scheme}://{uri.Host}";

        var configuration = new FtpConfiguration(
            baseAddress,
            Username,
            Password,
            portNumber,
            timeout: 10000);

        bool isConnected = await _ftpService.ConnectAsync(configuration);
...

With a properly formatted address, a Uri object is created to extract the base FTP address. A configuration object is then initialized, containing the server address, credentials, port number, and a timeout setting.
Using this configuration, the method calls _ftpService.ConnectAsync(), sending a request to the server. The success of the connection now depends on the server’s response.

Handling Connection Results

...
        if (isConnected)
        {
            StatusMessage = "Connected to FTP server";
            IsConnected = true;
            FtpItems = new ObservableCollection<FileSystemItem>(await _ftpService.ListDirectoryAsync(configuration));
            UpdateRemoteStats();
        }
        else
        {
            StatusMessage = "Failed to connect to FTP server";
            IsConnected = false;
        }
    }
    catch (Exception ex)
    {
        StatusMessage = $"Error: {ex.Message}";
        IsConnected = false;
    }
    finally
    {
        IsTransferring = false;
    }
}

If the connection succeeds, IsConnected = true, and the UI updates with a success message. A log entry confirms the connection, and the method retrieves the list of remote files, displaying them in the UI. The connection details are also saved for future use.
If the connection fails, an error message appears, and a log entry records the failure. Any unexpected errors—such as network issues—are caught, logged, and displayed. Finally, the method resets IsTransferring = false, indicating that the process has finished, regardless of the outcome.

Conclusion

In this article we explored the core aspects of working with FTP servers using FtpWebRequest in C#. We covered connection handling, retries, error management, and integration into an application. With intelligent retry mechanisms, this approach helps in building scalable and robust FTP-based applications. Whether you’re building a simple tool or a full-fledged FTP client, adopting these techniques will help you create a more resilient and user-friendly solution.

From XML to Word: simplifying conversion with FileConversionLibrary

Bohdan Harabadzhyu — Sat, 30 Nov 2024 13:15:00 +0000

FileConversionLibrary

File conversion can be a tedious task for developers, but FileConversionLibrary offers a basic solution for simple tasks. This library provides essential tools for converting CSV and XML files into formats like PDF, Word, YAML, and JSON. Available on NuGet and GitHub, it’s an easy-to-use tool for streamlining data processing workflows. In this article, I will focus specifically on converting XML to Word

Introduction to XML and Word Formats

XML (eXtensible Markup Language)

XML is a highly versatile format designed for storing and exchanging structured data. Developed by the World Wide Web Consortium (W3C) in 1998, it is widely used due to its compatibility and extensibility. Here are some technical details:

Key Features of XML:

Structure
- XML uses a tree-based hierarchy, starting with a single root element.
- Documents are composed of elements enclosed in tags.
Flexibility
- Users can define custom tags, adapting XML for various domains.
- Optional schemas like DTD (Document Type Definition) or XSD (XML Schema Definition) can enforce document structure.
Portability and Compatibility
- XML is platform-agnostic and supported across programming languages.
- It forms the basis for standards like SOAP (Simple Object Access Protocol) and SVG (Scalable Vector Graphics).
Drawbacks
- XML can be verbose, leading to large file sizes.
- Parsing XML is computationally heavier compared to alternatives like JSON.

Word Documents (.docx)

The .docx format is the modern standard for Microsoft Word documents, introduced in Office 2007 as part of the Office Open XML (OOXML) standard. It is designed to offer improved performance, compatibility, and extensibility.

Key Features of .docx:

File Structure
- .docx files are essentially ZIP archives containing multiple XML files and resources.
- Components include document.xml (content), styles.xml (styling), and settings.xml (document settings).
Content Representation
- Text and elements like tables are stored using a structured XML vocabulary.
- Relationships, such as those linking images or styles, are managed via rels (relationships) files.
Formatting and Extensibility
- Supports advanced formatting, including styles, fonts, and embedded media.
- Allows custom XML parts for metadata or structured content integration.
Advantages Over .doc
- Smaller file sizes due to ZIP compression.
- Improved interoperability with other software, thanks to its XML foundation.

Why Convert XML to Word?

XML is excellent for structuring data but isn’t user-friendly for non-technical audiences. Converting XML to Word bridges this gap, providing readable and editable documents for various use cases, such as:

Report Generation: Transform XML into formatted reports suitable for stakeholders.
Documentation: Automatically populate templates for manuals, invoices, or records.
Improved Accessibility: Enable non-technical users to view and edit data in a familiar format.

By converting XML to Word, you can unlock the potential of structured data while delivering it in a polished, user-friendly format.

Example XML

To demonstrate how FileConversionLibrary handles XML to Word conversion, we will use the following XML books.xml:

<?xml version="1.0"?>
<catalog>
    <book id="bk101">
        <author>Gambardella, Matthew</author>
        <title>XML Developer's Guide</title>
        <genre>Computer</genre>
        <price>44.95</price>
        <publish_date>2000-10-01</publish_date>
        <description>An in-depth look at creating applications
            with XML.</description>
    </book>
    <book id="bk102">
        <author>Ralls, Kim</author>
        <title>Midnight Rain</title>
        <genre>Fantasy</genre>
        <price>5.95</price>
        <publish_date>2000-12-16</publish_date>
        <description>A former architect battles corporate zombies,
            an evil sorceress, and her own childhood to become queen
            of the world.</description>
    </book>
 ...
</catalog>

After converting the XML, the data will be structured and formatted into a Word document. Each CD entry will appear as a table or a section, depending on your template and configuration.

Here’s an example of how it might look:

DocumentFormat.OpenXml: A Powerful Library for Office Document Manipulation

In the FileConversionLibrary, we leverage DocumentFormat.OpenXml to convert XML data into structured Word documents. DocumentFormat.OpenXml is an open-source library provided by Microsoft that enables developers to edit and process Office Open XML (OOXML) documents programmatically. This includes file formats such as .docx, .xlsx, and .pptx. The library eliminates the need to rely on Microsoft Office's COM Interop, making it lightweight and efficient for server-side or cross-platform applications.

Key Features of DocumentFormat.OpenXml

Standard Compliance
- The library adheres to the ISO/IEC 29500 standard, ensuring compatibility with modern Office applications. Works with .docx, .xlsx, and .pptx file formats introduced in Office 2007 and later.
Platform Independence
- Operates without requiring Microsoft Office installed on the host machine. Ideal for server-side applications, cloud-based systems, or non-Windows environments.
Efficient File Manipulation
- Enables the creation, reading, and modification of Office documents with fine-grained control. Supports advanced document structures, such as styles, metadata, tables, images, and charts.
XML-Based Architecture
- OOXML documents are structured XML files encapsulated in ZIP archives. Developers can directly interact with the XML nodes to customize or extract content.

XmlHelperFile: Simplifying XML Data Extraction

The XmlHelperFile class in FileConversionLibrary is designed to facilitate the extraction of data from XML files. This utility class provides an asynchronous method to read XML files and convert their contents into a structured format, making it easier to process and manipulate the data.

Key Features of XmlHelperFile

Asynchronous Operation.
The ReadXmlAsync method is asynchronous, ensuring non-blocking I/O operations which are crucial for performance in applications that handle large files or multiple concurrent tasks.
XML Parsing.
Utilizes XmlDocument and XmlNodeReader to parse the XML content, ensuring compatibility with standard XML structures.
DataSet Integration.
Converts the XML data into a DataSet, leveraging its powerful data manipulation capabilities to handle complex XML structures.
Structured Output.
Extracts headers and rows from the XML, returning them as a tuple containing an array of headers and a list of rows. Each row is represented as an array of strings, corresponding to the columns in the XML.

public static class XmlHelperFile
{
    public static async Task<(string[] Headers, List<string[]> Rows)> ReadXmlAsync(string xmlFilePath)
    {
        // Read the entire content of the XML file asynchronously
        var xmlContent = await File.ReadAllTextAsync(xmlFilePath);

        // Load the XML content into an XmlDocument
        var xmlFile = new XmlDocument();
        xmlFile.LoadXml(xmlContent);

        // Create an XmlNodeReader from the XmlDocument
        var xmlReader = new XmlNodeReader(xmlFile);

        // Create a DataSet and read the XML data into it
        var dataSet = new DataSet();
        dataSet.ReadXml(xmlReader);

        // Check if the DataSet contains any tables
        if (dataSet.Tables.Count == 0)
        {
            throw new Exception("No tables found in the XML file.");
        }

        // Get the first table from the DataSet
        var table = dataSet.Tables[0];

        // Extract the column names (headers) from the table
        var headers = new string[table.Columns.Count];
        for (var i = 0; i < table.Columns.Count; i++)
        {
            headers[i] = table.Columns[i].ColumnName;
        }

        // Extract the rows from the table
        var rows = new List<string[]>();
        foreach (DataRow row in table.Rows)
        {
            var rowData = new string[table.Columns.Count];
            for (var i = 0; i < table.Columns.Count; i++)
            {
                rowData[i] = row[i].ToString();
            }
            rows.Add(rowData);
        }

        // Return the headers and rows as a tuple
        return (headers, rows);
    }
}

IXmlConverter and XmlToWordConverter: Converting XML to Word

The IXmlConverter interface and XmlToWordConverter class in FileConversionLibrary provide a structured way to convert XML files into Word documents. This section explains how these components work together to achieve the conversion.

IXmlConverter Interface

The IXmlConverter interface defines a contract for converting XML files to other formats. It includes a single method, ConvertAsync, which performs the conversion asynchronously.

public interface IXmlConverter
{
    Task ConvertAsync(string xmlFilePath, string outputFilePath);
}

XmlToWordConverter Class

The XmlToWordConverter class implements the IXmlConverter interface to convert XML files into Word documents. It uses the XmlHelperFile class to read and parse the XML data, and the DocumentFormat.OpenXml library to create and manipulate the Word document.

public class XmlToWordConverter : IXmlConverter
{
    public async Task ConvertAsync(string xmlFilePath, string wordOutputPath)
    {
        try
        {
            // Read and parse the XML file using XmlHelperFile
            var (headers, rows) = await XmlHelperFile.ReadXmlAsync(xmlFilePath);

            // Create a new Word document
            using (var wordDocument = WordprocessingDocument.Create(wordOutputPath, WordprocessingDocumentType.Document))
            {
                // Add a main document part to the Word document
                var mainPart = wordDocument.AddMainDocumentPart();
                mainPart.Document = new Document();
                var body = mainPart.Document.AppendChild(new Body());

                // Add headers to the Word document
                var headerParagraph = body.AppendChild(new Paragraph());
                var headerRun = headerParagraph.AppendChild(new Run());
                headerRun.AppendChild(new Text(string.Join(" ", headers)));

                // Add rows to the Word document
                foreach (var row in rows)
                {
                    var paragraph = body.AppendChild(new Paragraph());
                    var run = paragraph.AppendChild(new Run());
                    run.AppendChild(new Text(string.Join(" ", row)));
                }
            }
        }
        catch (FileNotFoundException e)
        {
            // Handle file not found exception
            Console.WriteLine($"File not found: {e.FileName}");
        }
        catch (XmlException e)
        {
            // Handle invalid XML exception
            Console.WriteLine($"Invalid XML: {e.Message}");
        }
        catch (Exception e)
        {
            // Handle any other unexpected exceptions
            Console.WriteLine($"Unexpected error: {e.Message}");
        }
    }
}

How It Works

Reading and Parsing XML
The ConvertAsync method starts by calling XmlHelperFile.ReadXmlAsync to read and parse the XML file. This method returns the headers and rows extracted from the XML.
Creating a Word Document
The method then creates a new Word document using WordprocessingDocument.Create. This document is created at the specified wordOutputPath.
Adding Main Document Part
A main document part is added to the Word document, which contains the body of the document.
Adding Headers
A paragraph is created for the headers, and a run is added to this paragraph. The headers are joined into a single string and added as text to the run.
Adding Rows
For each row in the extracted data, a new paragraph is created, and a run is added to this paragraph. The row data is joined into a single string and added as text to the run.
Exception Handling
The method includes exception handling for FileNotFoundException, XmlException, and general exceptions to provide informative error messages.

Executing XML to Word Conversion

Here is an example of how to call the ConvertAsync method of the XmlToWordConverter class within the Main method:

class Program
{
    static async Task Main(string[] args)
    {
        // XML to Word Conversion
        var xmlToWordConverter = new XmlToWordConverter();
        await xmlToWordConverter.ConvertAsync(@"C:\Users\User\Desktop\books.xml", @"C:\Users\User\Desktop\output.docx");
        Console.WriteLine("XML to Word conversion completed.");

        ...
    }
}

Conclusion

In this article, we explored the process of converting XML files to Word documents using the FileConversionLibrary. My solution provides a basic, initial-level example of converting XML to Word. It serves as a basic example that can be taken as a starting point and improved upon.

Using CSCore for VoiceRecorder Application

Bohdan Harabadzhyu — Sun, 16 Jun 2024 10:12:52 +0000

Using CSCore for VoiceRecorder Application

CSCore is a powerful .NET library for audio processing and recording, offering a wide range of functionalities to work with sound. It supports multiple audio formats and devices, making it a go-to choice for developers working on audio-related applications. In this article I'll show you how to use CSCore using my simple VoiceRecorder application as an example, available on GitHub and SourceForge. I will focus on the core audio functionalities provided by CSCore and how they are integrated into my application. Note that we will not cover Avalonia UI or the MVVM design pattern.

Introduction to CSCore

CSCore offers a variety of features that make it an ideal choice for developers working on projects that require audio input and output. Some of the key features of CSCore include:

Support for multiple audio formats:
CSCore can handle various audio formats.
Audio capture and playback:
The library provides robust methods for capturing audio from different devices and playing back audio files.
Audio processing:
CSCore supports audio filtering, effects, and mixing, making it suitable for complex audio processing tasks.
Device management:
The library allows easy enumeration and selection of audio devices.

Key Features of VoiceRecorder

VoiceRecorder is an easy-to-use application designed to record audio from selected devices. It’s good for quick recordings and audio testing. The application simplifies the process of selecting an audio device, starting and stopping recordings, and saving the audio to a file. VoiceRecorder allows you to choose from three available filters to enhance your audio or record without any filters if you prefer.

Core Classes and Functionality

AudioDevice Class
The AudioDevice class is responsible for managing the audio devices available on the system. It uses the MMDeviceEnumerator from CSCore's CoreAudioAPI to list and select audio capture devices.

public sealed class AudioDevice : IDisposable
    {
        // Enumerates multimedia devices, allowing us to list   
        private MMDeviceEnumerator _mmdeviceEnumerator;
        private bool _disposed = false;

        public AudioDevice()
        {
            _mmdeviceEnumerator = new MMDeviceEnumerator();
        }

        // Retrieves a list of available audio capture devices
        public List<string> GetAvailableDevices()
        {
            return _mmdeviceEnumerator.EnumAudioEndpoints(DataFlow.Capture, DeviceState.Active)
                .Select(device => device.FriendlyName)  // Gets the friendly name of each device
                .ToList();
        }

        // Selects an audio device by its friendly name
        public MMDevice SelectDevice(string deviceName)
        {
            return _mmdeviceEnumerator.EnumAudioEndpoints(DataFlow.Capture, DeviceState.Active)
                .FirstOrDefault(device => device.FriendlyName == deviceName);
        }

        // Implements the Dispose pattern to release unmanaged resources
        private void Dispose(bool disposing)
        {
            if (!_disposed)
            {
                if (disposing)
                {
                    if (_mmdeviceEnumerator != null)
                    {
                        _mmdeviceEnumerator.Dispose();
                        _mmdeviceEnumerator = null;
                    }
                }
                _disposed = true;
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
    }

AudioRecorder Class
The AudioRecorder class handles the core recording functionality. It uses WasapiCapture for capturing audio and WaveWriter for writing the captured audio to a WAV file. The class also supports applying audio filters during recording.

public sealed class AudioRecorder : IDisposable
    {
        private WasapiCapture _capture; // Captures audio from the selected device
        private WaveWriter _writer; // Writes captured audio to a WAV file
        private bool _disposed = false;
        private SoundInSource _soundInSource; // Source for audio data from the capture device

        public IWaveSource CaptureSource => _soundInSource; // Exposes the current audio source

        // Starts recording audio from the specified device to a WAV file
        public void StartRecording(string outputFilePath, MMDevice device, IAudioFilter filter)
        {
            try
            {
                // Initialize audio capture with the selected device
                _capture = new WasapiCapture();
                _capture.Device = device;
                _capture.Initialize();

                // Create a SoundInSource to handle audio data
                _soundInSource = new SoundInSource(_capture) { FillWithZeros = false };

                // Apply filter if provided, otherwise use the raw source
                IWaveSource filteredSource;
                if (filter != null)
                {
                    filteredSource = filter.ApplyFilter((IWaveSource)_soundInSource);
                }
                else
                {
                    filteredSource = _soundInSource;
                }

                // Initialize WaveWriter to save the audio to a file
                _writer = new WaveWriter(outputFilePath, filteredSource.WaveFormat);

                // Buffer to hold audio data
                byte[] buffer = new byte[filteredSource.WaveFormat.BytesPerSecond / 2];

                // Event handler for when audio data is available
                _capture.DataAvailable += (s, e) =>
                {
                    int read;
                    while ((read = filteredSource.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        _writer.Write(buffer, 0, read);
                    }
                };

                // Start audio capture
                _capture.Start();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"An error occurred: {ex.Message}");
            }
        }

        // Stops the recording process and releases resources
        public void StopRecording()
        {
            try
            {
                _capture.Stop();
                _writer.Dispose();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"An error occurred: {ex.Message}");
            }
        }

        // Updates the audio source, allowing for dynamic changes during recording
        public void UpdateSource(IWaveSource newSource)
        {
            _capture.Stop();

            _soundInSource = newSource as SoundInSource;

            if (_soundInSource != null)
            {
                _capture.Start();
            }
            else
            {
                Console.WriteLine("newSource is not a SoundInSource");
            }
        }

        // Implements the Dispose pattern to release unmanaged resources
        private void Dispose(bool disposing)
        {
            if (!_disposed)
            {
                if (disposing)
                {
                    if (_capture != null)
                    {
                        _capture.Dispose();
                        _capture = null;
                    }

                    if (_writer != null)
                    {
                        _writer.Dispose();
                        _writer = null;
                    }
                }
                _disposed = true;
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
    }

Key Moments of MainWindowViewModel
The MainWindowViewModel class manages the interaction between the user interface and the main recording functionality. It maintains application state, processes user commands, and coordinates the recording process. However, in this article I want to focus only on the StartRecording method

public class MainWindowViewModel : ViewModelBase
{
        ...

     public void StartRecording(string deviceName, VoiceFilterViewModel filterViewModel)
  {
    // Generate a unique file path for the recording based on the device name
    string filePath = AudioFilePathHelper.GenerateAudioFilePath(deviceName);

    // Select the audio device that matches the provided device name
    var device = Device.SelectDevice(deviceName);

    // Check if a filter is provided and start recording with or without the filter
    if (filterViewModel != null && filterViewModel.FilterStrategy != null)
    {
        // Start recording with the provided filter strategy
        Recorder.StartRecording(filePath, device, filterViewModel.FilterStrategy);
    }
    else
    {
        // Start recording without any filter
        Recorder.StartRecording(filePath, device, null);
    }

    // Set the recording state to true to indicate that recording has started
    IsRecording = true;

    // If filters are applied and a filter is selected, apply the filter command
    if (IsFilterApplied && SelectedFilterViewModel != null)
    {
        ApplyFilterCommand();
    }
}

The StartRecording method in the MainWindowViewModel class initializes the recording process by selecting the appropriate audio device, determining whether to apply an audio filter, and starting the recording. It also updates the recording state and applies any selected filter if applicable.

Conclusion

In this article I did not touch on many topics: for example, working with filters and View classes. However, the purpose of this article was to show a clear example of using the CSCore library to capture and process audio. Using CSCore's robust features, we can create a simple yet powerful audio recording application. The focus on core classes and functions makes the application efficient and easy to understand, making it a suitable starting point for more complex audio processing projects. Happy coding!

Image Scraping with HtmlAgilityPack: A Practical Guide Using ConsoleWebScraper

Bohdan Harabadzhyu — Sun, 09 Jun 2024 10:11:00 +0000

Image Scraping with HtmlAgilityPack: A Practical Guide Using ConsoleWebScraper

Web scraping is a valuable tool for automating the collection of information from websites. My simple, open-source ConsoleWebScraper application, available on GitHub and SourceForge, demonstrates how to use the HtmlAgilityPack library to scrape images from web pages. This guide will focus on the image scraping capabilities of the application and provide an overview of its core functionality.

Introduction to HtmlAgilityPack

HtmlAgilityPack is a .NET library that simplifies HTML parsing, making it a favorite among developers for web scraping tasks. It provides a robust way to traverse and manipulate HTML documents. With HtmlAgilityPack, extracting elements like images and text from web pages becomes straightforward and efficient.

Why Scrape Images?

Images are a significant part of web content, used for various purposes such as visual data representation, marketing, and documentation. Scraping images can be useful for creating archiving content, or monitoring website changes. ConsoleWebScraper application serves as a simple example to demonstrate how you can automate this process using HtmlAgilityPack.

Key Features of ConsoleWebScraper

ConsoleWebScraper offers a few essential functionalities:
URL Input: Prompts the user to enter a URL and retrieves the HTML content.
HTML Parsing: Extracts inner URLs and images from the HTML content.
File Saving: Saves scraped URLs, images, and HTML content (with tags removed) to separate files for easy access and further analysis.

Additional Functionalities

Before diving into the core image scraping method, it's helpful to understand the broader functionality of the ConsoleWebScraper application, which includes several supporting methods and classes.
Another classes :
The Client class manages the interaction with the user and controls the application's flow. It listens for user commands and executes the appropriate actions
The Printer class provides simple methods to display the application's start page and main menu to the user.
The Controller class orchestrates the scraping process, managing user input, folder creation, and invoking the web scraper service methods.
The HtmlTags class provides a method to remove HTML tags from the content, leaving only the text.
The IWebScraperService interface defines methods for saving URLs, content, and images, which are implemented in the WebScraperService class.

The Core Method: SaveImagesToDoc

The heart of the image scraping functionality is encapsulated in the SaveImagesToDoc method. Let's dive deeper into this method to understand how it works.

public async Task SaveImagesToDoc(string fileName, string htmlContent, string baseUrl)
{
    // Create directory to save images
    Directory.CreateDirectory(fileName);

    // Load HTML content into HtmlDocument
    var doc = new HtmlAgilityPack.HtmlDocument();
    doc.LoadHtml(htmlContent);

    // Extract image URLs
    var images = doc.DocumentNode.Descendants("img")
        .Select(e => e.GetAttributeValue("src", null))
        .Where(src => !string.IsNullOrEmpty(src))
        .Select(src => new Uri(new Uri(baseUrl), src).AbsoluteUri)
        .ToList();

    // Initialize HttpClient
    using (HttpClient client = new HttpClient())
    {
        int pictureNumber = 1;
        foreach (var img in images)
        {
            try
            {
                // Download image as byte array
                var imageBytes = await client.GetByteArrayAsync(img);

                // Get image file extension
                var extension = Path.GetExtension(new Uri(img).AbsolutePath);

                // Save image to file
                await File.WriteAllBytesAsync($"{fileName}\\Image{pictureNumber}{extension}", imageBytes);
                pictureNumber++;
            }
            catch (Exception ex)
            {
                // Log any errors
                Console.WriteLine($"Failed to download or save image {img}: {ex.Message}");
            }
        }
    }
}

Step-by-Step Breakdown

Create Directory: The method starts by creating a directory to store the downloaded images.

Directory.CreateDirectory(fileName);

Load HTML Content: It then loads the provided HTML content into an HtmlDocument object from the HtmlAgilityPack library.

var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlContent);

Extract Image URLs: The method identifies all img elements in the HTML and extracts their src attributes. It converts these relative URLs to absolute URLs using the base URL of the web page.

var images = doc.DocumentNode.Descendants("img")
    .Select(e => e.GetAttributeValue("src", null))
    .Where(src => !string.IsNullOrEmpty(src))
    .Select(src => new Uri(new Uri(baseUrl), src).AbsoluteUri)
    .ToList();

Download and Save Images: Using HttpClient, the method iterates through the list of image URLs, downloads each image as a byte array, and saves it to the designated directory with an appropriate file extension. Errors during the download or save process are caught and logged.

using (HttpClient client = new HttpClient())
{
    int pictureNumber = 1;
    foreach (var img in images)
    {
        try
        {
            var imageBytes = await client.GetByteArrayAsync(img);
            var extension = Path.GetExtension(new Uri(img).AbsolutePath);
            await File.WriteAllBytesAsync($"{fileName}\\Image{pictureNumber}{extension}", imageBytes);
            pictureNumber++;
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to download or save image {img}: {ex.Message}");
        }
    }
}

Conclusion

The ConsoleWebScraper application demonstrates the fundamental use of the HtmlAgilityPack library to scrape images from web pages. Despite its simplicity, the tool offers basic functions, making it a good solution for entry-level tasks. By automating image extraction and storage, you can streamline your data collection efforts. Happy scraping!