Big data is a really big deal. Most of us are familiar with gigabytes (GB) and terabytes (TB) now that our flash drives and external hard drives are measured in these parameters. But do we really know how much data can be held on these devices?
If a person loads as many books as possible into a pickup truck and then stores them electronically, the data would consume only a 1 GB file. A 500 GB file could store all of the academic journal information currently contained on five library floors. Just ten 1-TB external hard drives could contain the entire printed collection of the largest library in the world, with more than 160 million items.
Today’s digital information revolution has catapulted us from terabytes into petabytes (PB) and exabytes (EB) of information. With only 3 PB of storage space, we could contain all US and Indian academic research libraries, while 200 PB of information would contain the equivalent of all printed material on the face of the earth. If we were to capture all words ever spoken by human beings, we would have the equivalent of only 5 EB of information.
Beginning in 2000, the University of California, Berkley, conducted a study to estimate how much information is produced every year. They estimated that in 1999, the world produced between 1 and 2 exabytes of new, unique information.
By 2002, they estimated that 5 EB of new information in print, film, magnetic and optical storage media had been added. The study concluded that the amount of new information stored on paper, film, magnetic and optical media had almost doubled in the last three years, and grew 30 per cent each year between 1999 and 2002.
Today, our information flows through electronic channels – telephone, radio, television and the Internet – which contained almost 18 EB of new information in 2002, three and a half times more information than is recorded in storage media. Ninety-eight per cent of this total volume of traffic was information sent and received through telephone calls, land lines and wireless.
According to former Google CEO Eric Schmidt, every two days we create as much information as we did from the dawn of civilisation through the year 2003 – an astonishing five exabytes of data. Most of this data is user-generated content consisting of pictures, instant messages, tweets and posts.
It follows that the growth of digital information has led to exponential growth in the use of databases by everyone – individuals, governments, and businesses, as well as good guys and bad guys. “Big Data” has turned into “Big Business!”
What is big data? Big data is the term used to refer to collections of extremely large and/or complex databases that become difficult to process using conventional database management tools. These databases now contain on the order of exabytes of data. Data sets, such as those used for meteorology, Internet activity, or traffic flow where ubiquitous information sensors are constantly monitoring their environments and gathering data, are some of the reasons that databases have grown exponentially over the past few years.
Big databases are superior to smaller databases when used to evaluate detailed trend data, as additional, detailed information can be derived easily by looking across the expanse of data. This level of trend detail may be missed when relying on smaller databases – all good reasons to collect and use big data.
Downloading wirelessly
However, two items are especially troubling from a personal, as well as a national security perspective. Big databases enable everyone to collect more information about you – both government and commercial organisations. From an espionage standpoint, anyone can now carry 128 GB of information – more than 64 million pages of text – on a USB drive in their pocket and walk out of the door, having downloaded the data from any accessible computer. More disturbing is the fact that this data could have been downloaded wirelessly!
Information is being harvested through a variety of programmes by Internet Service Providers, local wireless providers, Internet browsers and other programmes. Data collection is a multi-billion dollar industry. By advancing big data collection efforts, companies now have access to tens of thousands of data sources on a specific individual that can be compiled within milliseconds. This information provides personal information to marketers, politicians and businesses to predict consumers’ responses concerning healthcare, voter preference, and sales of everything from face creams to houses.
Google and other Internet and email providers mine your emails and searches for keywords in order to provide appropriate advertising. In many cases, companies don’t need to mine data. We freely give birth dates, names, and other information in order to receive access or “free” services. Are these really “free” when the company is using your data to resell?
Cyber security and protecting your data: How can you protect your data? Since most of the information in the world today is being generated by you, are you controlling your information? Consider these things before your next social media experience turns into a cyber nightmare:
1) Think before you post! Know what data you are providing. Ensure you don’t mind posting to the world. Remember, we provide most of the information ourselves which others can use to exploit us.
2) Know if it’s “electronic data” someone can get access to it. Never disclose personal information by email. If you provide personal information on websites, use only official, secured websites of the company with which you are conducting business.
3) Never click on a security related pop-up on your computer. End the session immediately.
4) Install and run a security scan on your computer daily to prevent infections.
(Iyengar is a distinguished Ryder Professor and Director, School of Computing and Information Sciences, Miami; Miller has been with US Air Force for over two decades and is Coordinator, Discovery Lab, Florida International University)
If a person loads as many books as possible into a pickup truck and then stores them electronically, the data would consume only a 1 GB file. A 500 GB file could store all of the academic journal information currently contained on five library floors. Just ten 1-TB external hard drives could contain the entire printed collection of the largest library in the world, with more than 160 million items.
Today’s digital information revolution has catapulted us from terabytes into petabytes (PB) and exabytes (EB) of information. With only 3 PB of storage space, we could contain all US and Indian academic research libraries, while 200 PB of information would contain the equivalent of all printed material on the face of the earth. If we were to capture all words ever spoken by human beings, we would have the equivalent of only 5 EB of information.
Beginning in 2000, the University of California, Berkley, conducted a study to estimate how much information is produced every year. They estimated that in 1999, the world produced between 1 and 2 exabytes of new, unique information.
By 2002, they estimated that 5 EB of new information in print, film, magnetic and optical storage media had been added. The study concluded that the amount of new information stored on paper, film, magnetic and optical media had almost doubled in the last three years, and grew 30 per cent each year between 1999 and 2002.
Today, our information flows through electronic channels – telephone, radio, television and the Internet – which contained almost 18 EB of new information in 2002, three and a half times more information than is recorded in storage media. Ninety-eight per cent of this total volume of traffic was information sent and received through telephone calls, land lines and wireless.
According to former Google CEO Eric Schmidt, every two days we create as much information as we did from the dawn of civilisation through the year 2003 – an astonishing five exabytes of data. Most of this data is user-generated content consisting of pictures, instant messages, tweets and posts.
It follows that the growth of digital information has led to exponential growth in the use of databases by everyone – individuals, governments, and businesses, as well as good guys and bad guys. “Big Data” has turned into “Big Business!”
What is big data? Big data is the term used to refer to collections of extremely large and/or complex databases that become difficult to process using conventional database management tools. These databases now contain on the order of exabytes of data. Data sets, such as those used for meteorology, Internet activity, or traffic flow where ubiquitous information sensors are constantly monitoring their environments and gathering data, are some of the reasons that databases have grown exponentially over the past few years.
Big databases are superior to smaller databases when used to evaluate detailed trend data, as additional, detailed information can be derived easily by looking across the expanse of data. This level of trend detail may be missed when relying on smaller databases – all good reasons to collect and use big data.
Downloading wirelessly
However, two items are especially troubling from a personal, as well as a national security perspective. Big databases enable everyone to collect more information about you – both government and commercial organisations. From an espionage standpoint, anyone can now carry 128 GB of information – more than 64 million pages of text – on a USB drive in their pocket and walk out of the door, having downloaded the data from any accessible computer. More disturbing is the fact that this data could have been downloaded wirelessly!
Information is being harvested through a variety of programmes by Internet Service Providers, local wireless providers, Internet browsers and other programmes. Data collection is a multi-billion dollar industry. By advancing big data collection efforts, companies now have access to tens of thousands of data sources on a specific individual that can be compiled within milliseconds. This information provides personal information to marketers, politicians and businesses to predict consumers’ responses concerning healthcare, voter preference, and sales of everything from face creams to houses.
Google and other Internet and email providers mine your emails and searches for keywords in order to provide appropriate advertising. In many cases, companies don’t need to mine data. We freely give birth dates, names, and other information in order to receive access or “free” services. Are these really “free” when the company is using your data to resell?
Cyber security and protecting your data: How can you protect your data? Since most of the information in the world today is being generated by you, are you controlling your information? Consider these things before your next social media experience turns into a cyber nightmare:
1) Think before you post! Know what data you are providing. Ensure you don’t mind posting to the world. Remember, we provide most of the information ourselves which others can use to exploit us.
2) Know if it’s “electronic data” someone can get access to it. Never disclose personal information by email. If you provide personal information on websites, use only official, secured websites of the company with which you are conducting business.
3) Never click on a security related pop-up on your computer. End the session immediately.
4) Install and run a security scan on your computer daily to prevent infections.
(Iyengar is a distinguished Ryder Professor and Director, School of Computing and Information Sciences, Miami; Miller has been with US Air Force for over two decades and is Coordinator, Discovery Lab, Florida International University)