DEV Community: Bohdan Harabadzhyu

From XML to Word: simplifying conversion with FileConversionLibrary

Bohdan Harabadzhyu — Sat, 30 Nov 2024 13:15:00 +0000

FileConversionLibrary

File conversion can be a tedious task for developers, but FileConversionLibrary offers a basic solution for simple tasks. This library provides essential tools for converting CSV and XML files into formats like PDF, Word, YAML, and JSON. Available on NuGet and GitHub, it’s an easy-to-use tool for streamlining data processing workflows. In this article, I will focus specifically on converting XML to Word

Introduction to XML and Word Formats

XML (eXtensible Markup Language)

XML is a highly versatile format designed for storing and exchanging structured data. Developed by the World Wide Web Consortium (W3C) in 1998, it is widely used due to its compatibility and extensibility. Here are some technical details:

Key Features of XML:

Structure
- XML uses a tree-based hierarchy, starting with a single root element.
- Documents are composed of elements enclosed in tags.
Flexibility
- Users can define custom tags, adapting XML for various domains.
- Optional schemas like DTD (Document Type Definition) or XSD (XML Schema Definition) can enforce document structure.
Portability and Compatibility
- XML is platform-agnostic and supported across programming languages.
- It forms the basis for standards like SOAP (Simple Object Access Protocol) and SVG (Scalable Vector Graphics).
Drawbacks
- XML can be verbose, leading to large file sizes.
- Parsing XML is computationally heavier compared to alternatives like JSON.

Word Documents (.docx)

The .docx format is the modern standard for Microsoft Word documents, introduced in Office 2007 as part of the Office Open XML (OOXML) standard. It is designed to offer improved performance, compatibility, and extensibility.

Key Features of .docx:

File Structure
- .docx files are essentially ZIP archives containing multiple XML files and resources.
- Components include document.xml (content), styles.xml (styling), and settings.xml (document settings).
Content Representation
- Text and elements like tables are stored using a structured XML vocabulary.
- Relationships, such as those linking images or styles, are managed via rels (relationships) files.
Formatting and Extensibility
- Supports advanced formatting, including styles, fonts, and embedded media.
- Allows custom XML parts for metadata or structured content integration.
Advantages Over .doc
- Smaller file sizes due to ZIP compression.
- Improved interoperability with other software, thanks to its XML foundation.

Why Convert XML to Word?

XML is excellent for structuring data but isn’t user-friendly for non-technical audiences. Converting XML to Word bridges this gap, providing readable and editable documents for various use cases, such as:

Report Generation: Transform XML into formatted reports suitable for stakeholders.
Documentation: Automatically populate templates for manuals, invoices, or records.
Improved Accessibility: Enable non-technical users to view and edit data in a familiar format.

By converting XML to Word, you can unlock the potential of structured data while delivering it in a polished, user-friendly format.

Example XML

To demonstrate how FileConversionLibrary handles XML to Word conversion, we will use the following XML books.xml:

<?xml version="1.0"?>
<catalog>
    <book id="bk101">
        <author>Gambardella, Matthew</author>
        <title>XML Developer's Guide</title>
        <genre>Computer</genre>
        <price>44.95</price>
        <publish_date>2000-10-01</publish_date>
        <description>An in-depth look at creating applications
            with XML.</description>
    </book>
    <book id="bk102">
        <author>Ralls, Kim</author>
        <title>Midnight Rain</title>
        <genre>Fantasy</genre>
        <price>5.95</price>
        <publish_date>2000-12-16</publish_date>
        <description>A former architect battles corporate zombies,
            an evil sorceress, and her own childhood to become queen
            of the world.</description>
    </book>
 ...
</catalog>

After converting the XML, the data will be structured and formatted into a Word document. Each CD entry will appear as a table or a section, depending on your template and configuration.

Here’s an example of how it might look:

DocumentFormat.OpenXml: A Powerful Library for Office Document Manipulation

In the FileConversionLibrary, we leverage DocumentFormat.OpenXml to convert XML data into structured Word documents. DocumentFormat.OpenXml is an open-source library provided by Microsoft that enables developers to edit and process Office Open XML (OOXML) documents programmatically. This includes file formats such as .docx, .xlsx, and .pptx. The library eliminates the need to rely on Microsoft Office's COM Interop, making it lightweight and efficient for server-side or cross-platform applications.

Key Features of DocumentFormat.OpenXml

Standard Compliance
- The library adheres to the ISO/IEC 29500 standard, ensuring compatibility with modern Office applications. Works with .docx, .xlsx, and .pptx file formats introduced in Office 2007 and later.
Platform Independence
- Operates without requiring Microsoft Office installed on the host machine. Ideal for server-side applications, cloud-based systems, or non-Windows environments.
Efficient File Manipulation
- Enables the creation, reading, and modification of Office documents with fine-grained control. Supports advanced document structures, such as styles, metadata, tables, images, and charts.
XML-Based Architecture
- OOXML documents are structured XML files encapsulated in ZIP archives. Developers can directly interact with the XML nodes to customize or extract content.

XmlHelperFile: Simplifying XML Data Extraction

The XmlHelperFile class in FileConversionLibrary is designed to facilitate the extraction of data from XML files. This utility class provides an asynchronous method to read XML files and convert their contents into a structured format, making it easier to process and manipulate the data.

Key Features of XmlHelperFile

Asynchronous Operation.
The ReadXmlAsync method is asynchronous, ensuring non-blocking I/O operations which are crucial for performance in applications that handle large files or multiple concurrent tasks.
XML Parsing.
Utilizes XmlDocument and XmlNodeReader to parse the XML content, ensuring compatibility with standard XML structures.
DataSet Integration.
Converts the XML data into a DataSet, leveraging its powerful data manipulation capabilities to handle complex XML structures.
Structured Output.
Extracts headers and rows from the XML, returning them as a tuple containing an array of headers and a list of rows. Each row is represented as an array of strings, corresponding to the columns in the XML.

public static class XmlHelperFile
{
    public static async Task<(string[] Headers, List<string[]> Rows)> ReadXmlAsync(string xmlFilePath)
    {
        // Read the entire content of the XML file asynchronously
        var xmlContent = await File.ReadAllTextAsync(xmlFilePath);

        // Load the XML content into an XmlDocument
        var xmlFile = new XmlDocument();
        xmlFile.LoadXml(xmlContent);

        // Create an XmlNodeReader from the XmlDocument
        var xmlReader = new XmlNodeReader(xmlFile);

        // Create a DataSet and read the XML data into it
        var dataSet = new DataSet();
        dataSet.ReadXml(xmlReader);

        // Check if the DataSet contains any tables
        if (dataSet.Tables.Count == 0)
        {
            throw new Exception("No tables found in the XML file.");
        }

        // Get the first table from the DataSet
        var table = dataSet.Tables[0];

        // Extract the column names (headers) from the table
        var headers = new string[table.Columns.Count];
        for (var i = 0; i < table.Columns.Count; i++)
        {
            headers[i] = table.Columns[i].ColumnName;
        }

        // Extract the rows from the table
        var rows = new List<string[]>();
        foreach (DataRow row in table.Rows)
        {
            var rowData = new string[table.Columns.Count];
            for (var i = 0; i < table.Columns.Count; i++)
            {
                rowData[i] = row[i].ToString();
            }
            rows.Add(rowData);
        }

        // Return the headers and rows as a tuple
        return (headers, rows);
    }
}

IXmlConverter and XmlToWordConverter: Converting XML to Word

The IXmlConverter interface and XmlToWordConverter class in FileConversionLibrary provide a structured way to convert XML files into Word documents. This section explains how these components work together to achieve the conversion.

IXmlConverter Interface

The IXmlConverter interface defines a contract for converting XML files to other formats. It includes a single method, ConvertAsync, which performs the conversion asynchronously.

public interface IXmlConverter
{
    Task ConvertAsync(string xmlFilePath, string outputFilePath);
}

XmlToWordConverter Class

The XmlToWordConverter class implements the IXmlConverter interface to convert XML files into Word documents. It uses the XmlHelperFile class to read and parse the XML data, and the DocumentFormat.OpenXml library to create and manipulate the Word document.

public class XmlToWordConverter : IXmlConverter
{
    public async Task ConvertAsync(string xmlFilePath, string wordOutputPath)
    {
        try
        {
            // Read and parse the XML file using XmlHelperFile
            var (headers, rows) = await XmlHelperFile.ReadXmlAsync(xmlFilePath);

            // Create a new Word document
            using (var wordDocument = WordprocessingDocument.Create(wordOutputPath, WordprocessingDocumentType.Document))
            {
                // Add a main document part to the Word document
                var mainPart = wordDocument.AddMainDocumentPart();
                mainPart.Document = new Document();
                var body = mainPart.Document.AppendChild(new Body());

                // Add headers to the Word document
                var headerParagraph = body.AppendChild(new Paragraph());
                var headerRun = headerParagraph.AppendChild(new Run());
                headerRun.AppendChild(new Text(string.Join(" ", headers)));

                // Add rows to the Word document
                foreach (var row in rows)
                {
                    var paragraph = body.AppendChild(new Paragraph());
                    var run = paragraph.AppendChild(new Run());
                    run.AppendChild(new Text(string.Join(" ", row)));
                }
            }
        }
        catch (FileNotFoundException e)
        {
            // Handle file not found exception
            Console.WriteLine($"File not found: {e.FileName}");
        }
        catch (XmlException e)
        {
            // Handle invalid XML exception
            Console.WriteLine($"Invalid XML: {e.Message}");
        }
        catch (Exception e)
        {
            // Handle any other unexpected exceptions
            Console.WriteLine($"Unexpected error: {e.Message}");
        }
    }
}

How It Works

Reading and Parsing XML
The ConvertAsync method starts by calling XmlHelperFile.ReadXmlAsync to read and parse the XML file. This method returns the headers and rows extracted from the XML.
Creating a Word Document
The method then creates a new Word document using WordprocessingDocument.Create. This document is created at the specified wordOutputPath.
Adding Main Document Part
A main document part is added to the Word document, which contains the body of the document.
Adding Headers
A paragraph is created for the headers, and a run is added to this paragraph. The headers are joined into a single string and added as text to the run.
Adding Rows
For each row in the extracted data, a new paragraph is created, and a run is added to this paragraph. The row data is joined into a single string and added as text to the run.
Exception Handling
The method includes exception handling for FileNotFoundException, XmlException, and general exceptions to provide informative error messages.

Executing XML to Word Conversion

Here is an example of how to call the ConvertAsync method of the XmlToWordConverter class within the Main method:

class Program
{
    static async Task Main(string[] args)
    {
        // XML to Word Conversion
        var xmlToWordConverter = new XmlToWordConverter();
        await xmlToWordConverter.ConvertAsync(@"C:\Users\User\Desktop\books.xml", @"C:\Users\User\Desktop\output.docx");
        Console.WriteLine("XML to Word conversion completed.");

        ...
    }
}

Conclusion

In this article, we explored the process of converting XML files to Word documents using the FileConversionLibrary. My solution provides a basic, initial-level example of converting XML to Word. It serves as a basic example that can be taken as a starting point and improved upon.

Using CSCore for VoiceRecorder Application

Bohdan Harabadzhyu — Sun, 16 Jun 2024 10:12:52 +0000

Using CSCore for VoiceRecorder Application

CSCore is a powerful .NET library for audio processing and recording, offering a wide range of functionalities to work with sound. It supports multiple audio formats and devices, making it a go-to choice for developers working on audio-related applications. In this article I'll show you how to use CSCore using my simple VoiceRecorder application as an example, available on GitHub and SourceForge. I will focus on the core audio functionalities provided by CSCore and how they are integrated into my application. Note that we will not cover Avalonia UI or the MVVM design pattern.

Introduction to CSCore

CSCore offers a variety of features that make it an ideal choice for developers working on projects that require audio input and output. Some of the key features of CSCore include:

Support for multiple audio formats:
CSCore can handle various audio formats.
Audio capture and playback:
The library provides robust methods for capturing audio from different devices and playing back audio files.
Audio processing:
CSCore supports audio filtering, effects, and mixing, making it suitable for complex audio processing tasks.
Device management:
The library allows easy enumeration and selection of audio devices.

Key Features of VoiceRecorder

VoiceRecorder is an easy-to-use application designed to record audio from selected devices. It’s good for quick recordings and audio testing. The application simplifies the process of selecting an audio device, starting and stopping recordings, and saving the audio to a file. VoiceRecorder allows you to choose from three available filters to enhance your audio or record without any filters if you prefer.

Core Classes and Functionality

AudioDevice Class
The AudioDevice class is responsible for managing the audio devices available on the system. It uses the MMDeviceEnumerator from CSCore's CoreAudioAPI to list and select audio capture devices.

public sealed class AudioDevice : IDisposable
    {
        // Enumerates multimedia devices, allowing us to list   
        private MMDeviceEnumerator _mmdeviceEnumerator;
        private bool _disposed = false;

        public AudioDevice()
        {
            _mmdeviceEnumerator = new MMDeviceEnumerator();
        }

        // Retrieves a list of available audio capture devices
        public List<string> GetAvailableDevices()
        {
            return _mmdeviceEnumerator.EnumAudioEndpoints(DataFlow.Capture, DeviceState.Active)
                .Select(device => device.FriendlyName)  // Gets the friendly name of each device
                .ToList();
        }

        // Selects an audio device by its friendly name
        public MMDevice SelectDevice(string deviceName)
        {
            return _mmdeviceEnumerator.EnumAudioEndpoints(DataFlow.Capture, DeviceState.Active)
                .FirstOrDefault(device => device.FriendlyName == deviceName);
        }

        // Implements the Dispose pattern to release unmanaged resources
        private void Dispose(bool disposing)
        {
            if (!_disposed)
            {
                if (disposing)
                {
                    if (_mmdeviceEnumerator != null)
                    {
                        _mmdeviceEnumerator.Dispose();
                        _mmdeviceEnumerator = null;
                    }
                }
                _disposed = true;
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
    }

AudioRecorder Class
The AudioRecorder class handles the core recording functionality. It uses WasapiCapture for capturing audio and WaveWriter for writing the captured audio to a WAV file. The class also supports applying audio filters during recording.

public sealed class AudioRecorder : IDisposable
    {
        private WasapiCapture _capture; // Captures audio from the selected device
        private WaveWriter _writer; // Writes captured audio to a WAV file
        private bool _disposed = false;
        private SoundInSource _soundInSource; // Source for audio data from the capture device

        public IWaveSource CaptureSource => _soundInSource; // Exposes the current audio source

        // Starts recording audio from the specified device to a WAV file
        public void StartRecording(string outputFilePath, MMDevice device, IAudioFilter filter)
        {
            try
            {
                // Initialize audio capture with the selected device
                _capture = new WasapiCapture();
                _capture.Device = device;
                _capture.Initialize();

                // Create a SoundInSource to handle audio data
                _soundInSource = new SoundInSource(_capture) { FillWithZeros = false };

                // Apply filter if provided, otherwise use the raw source
                IWaveSource filteredSource;
                if (filter != null)
                {
                    filteredSource = filter.ApplyFilter((IWaveSource)_soundInSource);
                }
                else
                {
                    filteredSource = _soundInSource;
                }

                // Initialize WaveWriter to save the audio to a file
                _writer = new WaveWriter(outputFilePath, filteredSource.WaveFormat);

                // Buffer to hold audio data
                byte[] buffer = new byte[filteredSource.WaveFormat.BytesPerSecond / 2];

                // Event handler for when audio data is available
                _capture.DataAvailable += (s, e) =>
                {
                    int read;
                    while ((read = filteredSource.Read(buffer, 0, buffer.Length)) > 0)
                    {
                        _writer.Write(buffer, 0, read);
                    }
                };

                // Start audio capture
                _capture.Start();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"An error occurred: {ex.Message}");
            }
        }

        // Stops the recording process and releases resources
        public void StopRecording()
        {
            try
            {
                _capture.Stop();
                _writer.Dispose();
            }
            catch (Exception ex)
            {
                Console.WriteLine($"An error occurred: {ex.Message}");
            }
        }

        // Updates the audio source, allowing for dynamic changes during recording
        public void UpdateSource(IWaveSource newSource)
        {
            _capture.Stop();

            _soundInSource = newSource as SoundInSource;

            if (_soundInSource != null)
            {
                _capture.Start();
            }
            else
            {
                Console.WriteLine("newSource is not a SoundInSource");
            }
        }

        // Implements the Dispose pattern to release unmanaged resources
        private void Dispose(bool disposing)
        {
            if (!_disposed)
            {
                if (disposing)
                {
                    if (_capture != null)
                    {
                        _capture.Dispose();
                        _capture = null;
                    }

                    if (_writer != null)
                    {
                        _writer.Dispose();
                        _writer = null;
                    }
                }
                _disposed = true;
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }
    }

Key Moments of MainWindowViewModel
The MainWindowViewModel class manages the interaction between the user interface and the main recording functionality. It maintains application state, processes user commands, and coordinates the recording process. However, in this article I want to focus only on the StartRecording method

public class MainWindowViewModel : ViewModelBase
{
        ...

     public void StartRecording(string deviceName, VoiceFilterViewModel filterViewModel)
  {
    // Generate a unique file path for the recording based on the device name
    string filePath = AudioFilePathHelper.GenerateAudioFilePath(deviceName);

    // Select the audio device that matches the provided device name
    var device = Device.SelectDevice(deviceName);

    // Check if a filter is provided and start recording with or without the filter
    if (filterViewModel != null && filterViewModel.FilterStrategy != null)
    {
        // Start recording with the provided filter strategy
        Recorder.StartRecording(filePath, device, filterViewModel.FilterStrategy);
    }
    else
    {
        // Start recording without any filter
        Recorder.StartRecording(filePath, device, null);
    }

    // Set the recording state to true to indicate that recording has started
    IsRecording = true;

    // If filters are applied and a filter is selected, apply the filter command
    if (IsFilterApplied && SelectedFilterViewModel != null)
    {
        ApplyFilterCommand();
    }
}

The StartRecording method in the MainWindowViewModel class initializes the recording process by selecting the appropriate audio device, determining whether to apply an audio filter, and starting the recording. It also updates the recording state and applies any selected filter if applicable.

Conclusion

In this article I did not touch on many topics: for example, working with filters and View classes. However, the purpose of this article was to show a clear example of using the CSCore library to capture and process audio. Using CSCore's robust features, we can create a simple yet powerful audio recording application. The focus on core classes and functions makes the application efficient and easy to understand, making it a suitable starting point for more complex audio processing projects. Happy coding!

Image Scraping with HtmlAgilityPack: A Practical Guide Using ConsoleWebScraper

Bohdan Harabadzhyu — Sun, 09 Jun 2024 10:11:00 +0000

Image Scraping with HtmlAgilityPack: A Practical Guide Using ConsoleWebScraper

Web scraping is a valuable tool for automating the collection of information from websites. My simple, open-source ConsoleWebScraper application, available on GitHub and SourceForge, demonstrates how to use the HtmlAgilityPack library to scrape images from web pages. This guide will focus on the image scraping capabilities of the application and provide an overview of its core functionality.

Introduction to HtmlAgilityPack

HtmlAgilityPack is a .NET library that simplifies HTML parsing, making it a favorite among developers for web scraping tasks. It provides a robust way to traverse and manipulate HTML documents. With HtmlAgilityPack, extracting elements like images and text from web pages becomes straightforward and efficient.

Why Scrape Images?

Images are a significant part of web content, used for various purposes such as visual data representation, marketing, and documentation. Scraping images can be useful for creating archiving content, or monitoring website changes. ConsoleWebScraper application serves as a simple example to demonstrate how you can automate this process using HtmlAgilityPack.

Key Features of ConsoleWebScraper

ConsoleWebScraper offers a few essential functionalities:
URL Input: Prompts the user to enter a URL and retrieves the HTML content.
HTML Parsing: Extracts inner URLs and images from the HTML content.
File Saving: Saves scraped URLs, images, and HTML content (with tags removed) to separate files for easy access and further analysis.

Additional Functionalities

Before diving into the core image scraping method, it's helpful to understand the broader functionality of the ConsoleWebScraper application, which includes several supporting methods and classes.
Another classes :
The Client class manages the interaction with the user and controls the application's flow. It listens for user commands and executes the appropriate actions
The Printer class provides simple methods to display the application's start page and main menu to the user.
The Controller class orchestrates the scraping process, managing user input, folder creation, and invoking the web scraper service methods.
The HtmlTags class provides a method to remove HTML tags from the content, leaving only the text.
The IWebScraperService interface defines methods for saving URLs, content, and images, which are implemented in the WebScraperService class.

The Core Method: SaveImagesToDoc

The heart of the image scraping functionality is encapsulated in the SaveImagesToDoc method. Let's dive deeper into this method to understand how it works.

public async Task SaveImagesToDoc(string fileName, string htmlContent, string baseUrl)
{
    // Create directory to save images
    Directory.CreateDirectory(fileName);

    // Load HTML content into HtmlDocument
    var doc = new HtmlAgilityPack.HtmlDocument();
    doc.LoadHtml(htmlContent);

    // Extract image URLs
    var images = doc.DocumentNode.Descendants("img")
        .Select(e => e.GetAttributeValue("src", null))
        .Where(src => !string.IsNullOrEmpty(src))
        .Select(src => new Uri(new Uri(baseUrl), src).AbsoluteUri)
        .ToList();

    // Initialize HttpClient
    using (HttpClient client = new HttpClient())
    {
        int pictureNumber = 1;
        foreach (var img in images)
        {
            try
            {
                // Download image as byte array
                var imageBytes = await client.GetByteArrayAsync(img);

                // Get image file extension
                var extension = Path.GetExtension(new Uri(img).AbsolutePath);

                // Save image to file
                await File.WriteAllBytesAsync($"{fileName}\\Image{pictureNumber}{extension}", imageBytes);
                pictureNumber++;
            }
            catch (Exception ex)
            {
                // Log any errors
                Console.WriteLine($"Failed to download or save image {img}: {ex.Message}");
            }
        }
    }
}

Step-by-Step Breakdown

Create Directory: The method starts by creating a directory to store the downloaded images.

Directory.CreateDirectory(fileName);

Load HTML Content: It then loads the provided HTML content into an HtmlDocument object from the HtmlAgilityPack library.

var doc = new HtmlAgilityPack.HtmlDocument();
doc.LoadHtml(htmlContent);

Extract Image URLs: The method identifies all img elements in the HTML and extracts their src attributes. It converts these relative URLs to absolute URLs using the base URL of the web page.

var images = doc.DocumentNode.Descendants("img")
    .Select(e => e.GetAttributeValue("src", null))
    .Where(src => !string.IsNullOrEmpty(src))
    .Select(src => new Uri(new Uri(baseUrl), src).AbsoluteUri)
    .ToList();

Download and Save Images: Using HttpClient, the method iterates through the list of image URLs, downloads each image as a byte array, and saves it to the designated directory with an appropriate file extension. Errors during the download or save process are caught and logged.

using (HttpClient client = new HttpClient())
{
    int pictureNumber = 1;
    foreach (var img in images)
    {
        try
        {
            var imageBytes = await client.GetByteArrayAsync(img);
            var extension = Path.GetExtension(new Uri(img).AbsolutePath);
            await File.WriteAllBytesAsync($"{fileName}\\Image{pictureNumber}{extension}", imageBytes);
            pictureNumber++;
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to download or save image {img}: {ex.Message}");
        }
    }
}

Conclusion

The ConsoleWebScraper application demonstrates the fundamental use of the HtmlAgilityPack library to scrape images from web pages. Despite its simplicity, the tool offers basic functions, making it a good solution for entry-level tasks. By automating image extraction and storage, you can streamline your data collection efforts. Happy scraping!