DEV Community

Cover image for Bigdata: A problem and a solution
 Radha
Radha

Posted on

Bigdata: A problem and a solution

We all like to be social in this world of technology, why? because it connects us from one place to another, without actually visiting!! connects us to the people without meeting them..!!

And we also like to surf in any web browsers like google, so that we can find the truth and the reality of everything we want know, or clear our doubts, watch videos play games etc.

We also like to be always updated in social media handle, sharing our pictures, videos, documents and everything we want, we actually like the way facebook, instagram or any other social media platform lets us to connect everywhere, let's us to store many and many pictures without our data storage problem and even don't delete them all....!! It's like storing everything and sharing without any tension of loosing them.

Cloud store our unique name and password without losing them!..

Umm... let's see our own example, our own laptop, our laptop consist of certain fixed amount of memory and storage, and one it's filled then we've to delete it or transfer it to other device.

But is that also the case with big big multinational social companies? Like google, facebook etc!!? Do they also have data storage problem..??? Do they also need more and more space to and storage to store every data??

Well, the answer is YES!.. everyone is suffering from data storage problem!!!

But, it is our case that we have certain fixed amount of storage and we've only data storage problem!!... But actually, here everyone is facing the same issue.....The issue of data storage!

So what!!, they're BIG MULTINATIONAL COMPANIES!!, they've too much of money, they can buy too much amount of storage!!

Yes, of course they can easily buy the storage, but till WHEN??? .......not always..!!

This is not an actual solution of data storage problem!!

So what the actual problem is....??? Do they only suffer from data storage problem only???The answer is NO!!!........

There's 2 MAIN PROBLEM that every multinational company suffer from and also a problem of cost..!!

That is:

-Volume: means size of data, and storage

-Velocity: Time consumed in transferring or receiving of data, in simple term, it is also known as I/O(input/ output), input output processing operation.

We, call this problem as "BIGDATA" problem, bigdata is like an umbrella of problems, that most of the companies face in day to day life...!!

In this world, full of technology, the company which is receiving huge number of data day-to-day and are able to manage is the leading company!!! There're many problems that they too face, but they're able to manage it and hence they are at top, Ever thought why Facebook users are more? why google search engine is used more? This is all due to their management and use of new technologies along with huge number of problems, and the way they manage and the technology they use matter lot to us!!! Because in this era of very tough competition, being and able to stay at same top position is tough and also it is getting more tougher as the day passes!!!!
Let's now see, what actually is big data? and how does it work??

We need a technology which can solve our problem of velocity and volume efficiently, and this concept is known as "Distributed Storage" concept.

It's like solving both the problem at same time and efficiently.

It is some what like distributing small small problems so that it can be solved easily, like as you can see in the diagram:

Diagram
It shows that a single huge server/pc distributes or splits it's problem to 5 other pc's, which concludes that, 50GB storage is distributed equally in 5 other pc's which resolves our data storage problem, and secondly, suppose it takes 50 min to transfer 50 GB of data, so after this distribution our second problem also resolved, that is transferring data or i/o processing operation of data, which means now a single pc will take 10 minutes to transfer 10 gb of data which means overall 5 pc's at same time will take 10 minutes and so our 40 extra minutes time is saved which means our data speed is also increased using distributed concept.

Img
Which solely means in simple terms : total no. of device/total storage == two main problems resolved!!! More and more pc/hdd we use more our speed will increase and and volume will get distributed!

Their are many more problems which they are facing, and most of their problem is resolved by big data and hadoop.

Well, it was a small article on how and which two main issues are being faced by multinational companies..!!

That was all in this article,

Thanks for reading!!!

Top comments (0)