Memory management techniques to reduce memory consumption and improve performance.

pratheek87 — Wed, 06 Oct 2021 09:54:31 +0000

Every memory allocation comes with a overhead of book keeping data. Also, every memory request results in a call to the operating system. Imagine a scenario where you would need to create millions of tiny objects over the entire time of the application usage. In environments where memory is a serious constraint it could lead to application failures because of the memory consumed by the book keeping data.

In this post, I would like to share a common technique used to handle memory management to address the above concerns.

Key concepts:

Creating a memory pool with the required size. Future allocations would be made from this pool rather than directly allocating from the OS. This would also avoid memory consumed by book-keeping.
Overload the new operator to handle memory allocation requests using this pool.
Make the memory allocator class to be used across by multiple classes rather than individual classes.

Code Snippet:

Write a class that creates a memory pool of required size. The key function here is the createpool method that calls malloc to allocate memory of required size and creates a list of FreeList pointers.

class memallocator
{
public:
    memallocator(size_t reserve) : mReserve(reserve) {}

private:
    size_t mReserve;
    FreeList* mBegin = nullptr;
    stack<FreeList*> freeptrs;
public:

    void* allocate(size_t s)
    {
        if (mBegin == nullptr)
        {
            if (!freeptrs.empty())
            {
                void* ptr = freeptrs.top();
                freeptrs.pop();
                return ptr;
            }
            else
                createPool(s);
        }
        FreeList* pFreeList = mBegin;
        mBegin = mBegin->next;

        return pFreeList;
    }

    void deallocate(void* ptr, size_t s)
    {
        freeptrs.push(reinterpret_cast<FreeList*>(ptr));
    }


    void createPool(size_t s)
    {
        int total_size = s * mReserve;
        FreeList* pMem = reinterpret_cast<FreeList*>(malloc(total_size));
        FreeList* temp = pMem;

        //Create the list of pointers.
        for (auto i = 0; i < mReserve-1; ++i)
        {
            temp->next = reinterpret_cast<FreeList*>(reinterpret_cast<char*>(temp) + s);
            temp = temp->next;
        }

        temp->next = nullptr;
        mBegin = pMem;
    }
};

FreeList is defined as a struct below that would be used to keep track of the next pointer.

struct FreeList
{
    FreeList* next;
};

In the custom class overload the new and delete operators to make use of the new allocator we have created.

class MyCustomClass
{

    string mptrData;
    double d;

public:
    static memallocator allocator;
    MyCustomClass(const char* data, int s, double val) : d(val) {
        mptrData = data;
    }

    static void* operator new(size_t s)
    {
        return allocator.allocate(s);
    }

    static void operator delete(void* ptr, size_t s)
    {
        allocator.deallocate(ptr, s);
    }

    string sendData()
    {
        return mptrData + " " + to_string(d);
    }
};

The below code is needed to initialize the allocator object. We are reserving space for about 96000 objects.

memallocator MyCustomClass::allocator{ 96000 }; // Reserve space for 24 objects.

** Note **: 96000 was used for testing purposes only.

Let us write the main function to test

int main()
{
    const char* data = "This is test object : ";
    long long int begin = GetTickCount64();

    for (auto i = 0; i < 240000; ++i)
    {
        auto* mc = new MyCustomClass(data, strlen(data), i);
    }

    long long int end = GetTickCount64();
    double elapsed = (end - begin);

    cout << "Time measured: %.9f miliseconds.\n" <<  elapsed << endl;

Key takeaways:

When tested with the memory allocator we have written along with the default we could see that our memory allocator was efficient slightly with time (since my CPU is quite powerful I guess :) ) but very efficient in terms of the memory consumed. For the above sample it was around 10K less taken by our memory allocator which is quite a decent gain.
This is not industry standard approach rather a simple technique that needs to be customized for specific requirements.

Please reach out to me for any further questions.

C++ - Handling memory allocation failures when using the new operator.

pratheek87 — Thu, 30 Sep 2021 08:17:50 +0000

The below code is to demonstrate how to handle memory allocation failures when creating objects of custom class as well as existing data types.

Key concepts:

Using the set_new_handler function.
Global handlers vs class specific handlers to handle memory allocation failures.

set_new_handler function

The set_new_handler method is used to set/change the custom handler function invoked when there are memory allocation failures. If not set, the new operator invokes the default handler on failure which eventually throws bad allocation error.

Syntax:

new_handler set_new_handler(new_handler p) throw();

Parameter:

A funtion poiner that receives and returns nothing.

Return value:

The value of the current new - handler function if this has already been set by this function previously, or a null
pointer if this is the first call to set_new_handler(or if it was reset by a previous call).

Note:

new_handler is a typedef for a function pointer that receives and returns nothing.

Global handlers vs class specific handlers

Code Snippet:

The below functions are used as handlers for global allocation and class object allocation

//Global
void nomemory()
{
    cerr << "No memory available" << endl;
    throw bad_alloc();
}

//Class object
void ctest_nomemory()
{
    cerr << "No memory available for allocating the class object" << endl;
    throw bad_alloc();
}

The below test class overloads the new operator.
The currentHandler member points to the function that needs to be invoked when the allocation fails for this class.

class CTest
{
public:
    static new_handler set_new_handler(new_handler p);
    static void* operator new(size_t s);

private:
    static new_handler currHandler;

    double x[100000000];
    double x1[100000000];
    double x2[100000000];
    double x3[100000000];
    double x4[100000000];
};

//The below function overrides the std::set_new_handler method.
//The function returns the existing handler method and sets the 
//passed function as the new handler.
new_handler CTest::set_new_handler(new_handler p)
{
    new_handler old_handler = currHandler;
    currHandler = p;
    return old_handler;
}

The below function overloads the new operator.
The std::set_new_handler function sets the global handler.
The function later sets the old handler back before the function exits.

void* CTest::operator new(size_t s)
{
    new_handler global_handler = std::set_new_handler(currHandler);

    void* memory;
    try
    {
        memory = ::operator new(s);
    }
    catch (std::bad_alloc&)
    {
        std::set_new_handler(global_handler);
        throw;
    }
    std::set_new_handler(global_handler);

    return memory;
}

//Initialize the static object 
new_handler CTest::currHandler;

In the main method first we can see the usage of the global handler and the next snippet for the class object:

int main()
{
    //Sets the global new handler.
    set_new_handler(nomemory);

    try
    {
        for (; ; )
            double* d = new double[100000000];
    }
    catch (bad_alloc)
    {
        cout << "allocation failed" << endl;
    }

    //Sets the class CTest new handler.
    CTest::set_new_handler(ctest_nomemory);
    try {
        CTest* testObj = new CTest;
    }
    catch (std::bad_alloc&)
    {
        cout << "allocation failed" << endl;
    }

    return 1;
}

Output:

No memory available
allocation failed

No memory available for allocating the class object
allocation failed

DEV Community: pratheek87

Memory management techniques to reduce memory consumption and improve performance.

Key concepts:

Code Snippet:

Key takeaways:

C++ - Handling memory allocation failures when using the new operator.

The below code is to demonstrate how to handle memory allocation failures when creating objects of custom class as well as existing data types.

Key concepts:

set_new_handler function

Syntax:

Parameter:

Return value:

Note:

Global handlers vs class specific handlers

Code Snippet:

Output: