Example of Big Data
A computer science student asked: “My friend who works in the industry told me that Big Data is only a concept that will happen in the future and someday be used in the industry. I am confused, is it something that already being used or is it still a new concept? Please advice?”
Answer: Big Data is NOT a new concept; it has been used for many years especially in financial analysis and stock market trading. Today, it is widely used in many industries from retails, banking, to pharmaceuticals and biotechnology, etc. In the past, the use of Big Data was limited to large companies because it was very expensive as it required powerful computers with large storage capacity. Today advancing of computer and storage technology has made Big Data affordable so its use is getting more popular.
If you look back in history, before the invention of Personal Computer (PC), only very large companies can afford the multi-million dollars mainframe computers. Apple and Microsoft have changed that by putting a computer in every home. The same thing is happening today with Big Data, as the cost of high speed processing and large storage decrease, more companies can afford to use them and Big Data is becoming a fast growing area. Industry analysts predicted that in the past, software has helped thousands of developers become millionaires and billionaires then this cycle will repeat with Big Data and soon many Data scientists, Data architects, Data analysts will become millionaires and billionaires.
Today company can access a vast amount of data both inside and outside the company and leverage them to improve business. The concept is simple: Collect a lot of data, in all kinds of formats from many sources, analyze and organize them to provide essential information for management’s decisions. However the implementation is NOT that simple, it requires special skills. Big Data requires four types of skill: The data collection skills (Database administration and Data Mining skills) to collect, format, organize and store them in the database or data warehouse. The analytics skills (Mathematics, statistics and integration skills) to analyze these data and translate them into useful information. The Interpreting skills (Business Intelligence, visualization, virtualization, and graphic skills) to representing this information in presentations, graphs, charts or metrics as used in OLAP. The Leveraging skills (Mathematics, statistics, and predicting skills) to create prediction, forecasting reports.
Walmart, the largest retail store in the world has been using Big Data to improve its operational efficiency since 1990s. Their managers understand that by having all types of data, they can improve its supply chain, limit excess inventory, reduce operational costs, and lower its prices. Within a short time, Walmart has eliminated many of its competitors by having lower prices due to its operational efficiency. Before Walmart used Big Data, there was thousands of big retail companies in the world, today the number dropped to less than a hundred. Walmart is now the largest retail company in the world with thousands of stores and millions of employees. The advantage that Walmart has over its competitors is its sophisticated information systems and the use of Big Data. Here are few examples:
- Dynamic price optimization: By using Big Data and business intelligence techniques, Walmart can change the price of its products in each store automatically. By collect market data from all over the world, including competitor pricing, supply chain, inventory data, and consumer behavior data, Walmart can adjust any prices to maximize sales and increase profits. When a competitor has lower price, Walmart computers immediately drop its price by few percent more to guarantee that its price is the lowest.
- Sales analysis and support: Walmart collects all sales information. It does not keep a large inventory but when certain items fell below a minimum amount, its computer system automatically place an order to the manufacture to ship additional products to specific store. By using Big Data analytics, the company knows in details the market trends in geographically dispersed areas to optimize its in-store products.
- Product placement analysis: All Walmart stores have video cameras to collect customers’ behavior as they shop. By analyzing video data and store layout, Walmart knows where to place their products to encourage more buying behavior. A very low price product placed at the end of the aisle will force customers to go all the way to the end, passing many other product displays. Every time they stop and look at these displays, their behavior signals interests. The data is collected and analyzed carefully to determine trends. By using Big Data analytics, Walmart understands which products may attract large numbers of customer and take action by change the price as special sales to promote more customers to come to their stores.
Of course, Walmart is just one example as more companies and industries are quickly adopting Big Data analytics for their competitive advantages. Although demand is high but currently few schools are teaching these skills. It is predicted that it may need at least another decade or so to have enough Data scientists, Data analysts or Data engineers for the industry.
Sources
- Blogs of Prof. John Vu, Carnegie Mellon University
