Blog

With the advent of Hadoop, the question of whether it would take over certain data warehousing functions or replace data warehouses altogether has become a hot-button topic of discussion. Finding an answer to this question really depends on how a data warehouse is viewed by individual organizations.

If I were to tell you that companies around the world are prioritizing structured data analytics initiatives, you wouldn’t think twice about it. Given the progressive climate of big data and analytics, it seems like that would be a fair statement to make. Especially as many businesses continue to count on their relational databases, ERPs, CRMs, and other data management systems to organize, structure, store, and define their structured data sets to run more meaningful analytics.

One of the most important lessons we are learning in the Big Data and Analytics Age is that simply having access to innovative technology isn’t enough to improve business outcomes. Technology needs the proper human input and interaction to elicit the types of results business leaders are expecting. Technology shouldn’t be counted on as a magic wand that will always deliver upon request.

IBM Watson has been getting a lot of publicity for helping organizations tackle real-world issues on a national and global scale.

We are entering an age in which corporate predictions and decisions can’t be made without sound analytical backing. Without data credibility, even the most promising ideas and initiatives will be shot down. In an interview with the Harvard Business Review, Tom Davenport (author of the book, Analytics at Work: Smarter Decisions, Better Results) expounded on this point:

Apache Spark is a game changer for big data analytics. In many ways, it will democratize analytics on big data. Spark will do this by making big data analytics accessible to a much larger group of data scientists (and analysts) via a simple programming API and familiar tools such as SQL, Python and R. Being open source, it will also allow many companies to embrace it with a smaller initial capital investment. Spark is also being embraced by IBM and other large analytic software providers as the engine that will drive their big data applications. The primary benefits of Spark are the ability to scale analytics to massive data sets and that Spark is a significantly easier to use platform (than Hadoop + Map Reduce + custom code) that will allow data scientists to be far more productive. Rapid development often results in a faster ROI.

The fear factor stems from the idea that data is being exposed to the public. Also, that providers are the ones in control of the primary source data. Entrusting a cloud vendor with the keys to the data castle is intimidating no matter how you look at it.

Imagine any milestone in your lifetime when fear potentially stood in the way of watershed success. Switching career paths, moving to different parts of the country, or even something as simple as buying a new car. If the fear of change or failure prevented you from making a move, could you imagine what would happen if you stayed stagnant? Chances, are you couldn’t have achieved a certain level of progress.