Big Data and its impacts
A Software Engineering student wrote to me: “Technologies change fast and it is difficult to learn everything. In your opinion, which ones is the most important technology? If I can put my effort to learn one or two technologies, what would they be? Please advise.”
Answer: Although it is difficult to predict which technology will be the most important in the next five or ten years. However my own opinion, Cloud computing, Big Data analytics and Computing security are the three most promising technologies. Since I conduct research and teach Big data analytics at CMU, I have a bias to favor this technology. I think what the industry is doing today with Big data is only the beginning. Since there is a critical shortage of skilled data scientists to collect, organize, process, and analyze, this technology has not reach the full potential yet. However, when there are more data scientists, the situation will change and the technology could explode. The fact is last year the industry had spent several billions of dollars in this technology and there is a race to see who will be first among companies such as IBM, Oracle, GE, SAS, and SAP as they invest billion dollars to develop software and hardware that perform Big data analytics.
GE is taking data from everything that they built i.e., Jet engines, power plants, locomotive engines, hospital equipments etc. integrate them with data from the web to extract valuable information. Basically, it is the integration of big data and cloud computing to monitor and predict what is happening inside GE products. By having such data, GE can help customers identify maintenance problems before they occur, improve fuel efficiency, and make other operational improvements that could add up to trillions of dollars in business. A senior manager said: “Our goal is to make our machines more intelligent and getting valuable data to customers in real-time.” The data that GE collects through its equipments are gigantic. For example a power plant turbine generates 500 gigabytes of data daily. With 12,000 of them are being used to power a quarter of the world’s electricity, the business opportunity is valued at several trillion dollars. A manager told me: “Even we use a conservative estimate of improving only one percent in our machinery (aviation, power plant, healthcare, railroad), we can make additional $300 billion. If we can improve our jet engines only one percent in fuel efficiency that could mean saving $2 billion a year for the airlines. That is why big data is “big business” as the industry can envision a $30 trillion opportunity in the next ten year. It is the biggest technology impact ever, bigger than oil, bigger than manufacturing, and bigger than the Internet. If a company can seize this opportunity, it can be ten times bigger than Google, Apple, and Facebook combine. That is why we called it “Big data”
A Wall Street Analyst wrote: “We believe Big data will be a major change in the same way that the Internet has changed the industries in the past decade. Before the Internet, Walmart was the largest retail company in the world but now it has a competitor: Amazon. com. In the next 10 years, the changes that we saw in the Internet will happen in the industries with Big data. It will impact every industry, every business, and everything and that is why investing in Big data is the priority of many Wall Street investors.”
For Big data analytics, industry needs “Smart computer” who can think like people but much faster. That is why IBM creates a special computer called “Watson” that combines artificial intelligence, natural language processing to process Big data and making decisions quickly. By “thinking” and make “decision faster” than human, smart computers can be used in industries such as healthcare, finance, stock markets, transportation and robotics. With special computer that can think and act like human, it can help drive cars instead of people. Some self-driving cars are available now (Several Google executives already have them) and soon it will be available to the public as more people want cars that can take them to wherever they want to go without have to drive it. Last year, when Taxi companies invested billion dollars in this technology, taxi drivers were angry as they can see that in few more years, they may not have jobs. But software engineers and programmers were happy when Taxi companies are planning to hire thousands of them to program and manage their self-driving Taxi fleets.
I think “Big data” will grow fast and impact the industry in the next few years. I think 2014 is the beginning of a data-focus era where data is the new assets. Of course since you ask, my advice is to put your efforts in learning more about this new technology as I think it has a very good future.
Sources
- Blogs of Prof. John Vu, Carnegie Mellon University
