It is no secret that analytics and planning projects are based on data. Typically projects involve wrangling some historical datasets, each with their own peculiarities, and then adding in some projection or prediction. Because data is essential, it is also a common stumbling block as business users consider beginning a new analytics and planning project.
Clients have expressed any number of data concerns to me over the years, but a handful of objections seem to float to the top most often. These concerns are typically logical but are actually not going to prevent them from implementing a new system or process. Analytics, regardless of the platforms being integrated, is about the details of the use case at hand. When viewed correctly through the lens of a use case, these data concerns almost always disappear. For this reason I think of these issues as "myths" that we have to dispel in order to advance and initiative.
The list below details the most common myths and why they won't stop you from successful execution on an analytics and planning project.
Myth #1: We don't have enough data.
How much is enough? That depends on the use case being pursued. Over hundreds and hundreds of past projects, only in two cases has a client actually not had enough data to achieve their goals. In both of those situations, the needed data had only just started being collected so the issue truly was low volume. Unless you've just started collecting data, the organization has enough historical data to begin working. Forecasting models can be more accurate with more data (to a point!) but a shorter time horizon shouldn't hold you back from making progress. Start now and get better over time as data accrues.
Myth #2: We have too much data.
This isn't a problem, this is a blessing! Certain use cases such as price optimization and demand planning work better with more historical data. Usually when a client raises this concern, the culprit is the level of detail. For a retailer or distributor, the base level data is by product by day and by customer or store. Across a whole company and large product catalog, that can be an intimidating amount of data. However, today's planning and analytics systems can scale to accommodate the required data. The key is determining what the required data is-- and often users only need a summarized level of data.
Myth #3: Our data warehouse has everything we need.
This just simply is never true. Structured data warehouses are great assets but a planning or analytics project typically needs data in a different format or needs to include data not in the warehouse married up to data from the warehouse. Its never as simple as just pulling the data in, but certain data like base-level transactions or historical sales data can usually be sourced from a warehouse easily.
Myth #4: We can never get away from Excel.
Even the largest companies' processes include a spreadsheet somewhere along the way. It seems strange but I've found it to be consistently true. Why is that? There are several reasons, each of which can be addressed during a project implementation.
Myth #5: We can't maintain this ourselves.
For the past decade much of enterprise software purchasing has been driven by those outside the domain of the IT department. Line of business users are in the driver's seat in system selection and implementation. Independence and the ability to manage the system is paramount, but when it comes to data and integration, business users get cold feet. There's a fear of the unknown and integration sounds really complex.
The reality is that maintaining system and data integrations doesn't have to be complex. Maintenance does need to be considered during the design and tool selection phase, however. Think "People, Process, and Technology" relative to maintenance to be sure you create something that is effective and sustainable.
Myth #6: Our data is dirty or poor quality.
The truth is everyone's data is dirty to some degree. Experience has taught us what matters and what doesn't. With today's tools it is easy to build pipelines and processes that standardize and aggregate data for quality and consistency. But there is no magic bullet-- project stakeholders often have to put in the time and effort to map codes across systems and manage hierarchies.
Don't use data as a roadblock for an integration or project. The challenges with existing data can be overcome with design, tools, and process. For more information, look for more from QueBIT on relational datahubs and register for our upcoming webinar - Busting Myths about Data Prep and Integration.