In the present information age, big data presents incredible opportunities - smarter cities, better managed companies, faster medicine - but just as many challenges. The world now contains unimaginably vast amounts of digital information, which is growing exponentially. Managed well, this data can be used to engineer new engines of economic value, unlock scientific breakthroughs, and hold politicians accountable. Managed poorly, it can cause great harm. The financial crisis showed that complex models that analyze large quantities of data do not always reflect financial risk in the real world. The financial crisis was sparked by big data—and there will be others. But the data deluge will also generate millions of new ideas for how to solve big problems, build new markets, and expand existing ones. In the past couple of years, big data has essentially become a buzz word in the enterprise technology world. However, people have forgotten that big data has been around and put to good use for a long time. In fact, Big Data is as old as the US Constitution.
Image Source: http://steveboese.squarespace.com/journal/2012/3/12/big-data-coming-to-a-staff-meeting-near-you.html
During the 1880 Census hundreds of clerks transferred data manually. Herman Hollerith, a statistician, thought that there must be a mechanical way of getting this job completed. Thus the U.S. census of 1890 which got completed in two years was a milestone in the history of modern data processing. This was the start of the age of mechanized data handling.
The Economist on IBM’s celebration of its 100th birthday: “In 1886 Herman Hollerith, started a business to rent out the tabulating machines he had originally invented for America’s census. Taking a page from train conductors, who then punched holes in tickets to denote passengers’ observable traits (e.g., that they were tall, or female) to prevent fraud, he developed a punch card that held a person’s data and an electric contraption to read it. The technology became the core of IBM’s business when it was incorporated as Computing Tabulating Recording Company (CTR) in 1911 after Hollerith’s firm merged with three others.”
In the late 1950s American Airlines pioneered the first airline reservation system which would allow its offices real-time access to flight information and automation of the booking and ticketing processes. They called it Sabre (Semi-Automatic Business-Related Environment). It was developed and launched in 1964 automating the reservation system and helping the airline to keep its inventory accurate in real time. IBM then stepped in and assisted American airlines to improve the system and extend its services to other agents and airlines. This system has been processing millions of transactions and saving companies like Travelocity.com money and offering reliability.
Extracting value from big data is gathering momentum across every industry. But while today’s media hype surrounds predictive analytics and new database technologies, one shouldn’t forget that there exists technologies out there which have been harnessing big data for years.
With the increase in enterprise data volumes and social networking, do you think big data is a recent phenomenon? Tell us your thoughts in the comments section!
Somaditya Sinha is a 2nd year Masters student in the Information Management program at Syracuse University. He is interested in management of large data sets and the design of data management systems. Contact him at email@example.com or follow him on twitter @somsinha86