It is no secret that analytics and planning projects are based on data. Typically projects involve wrangling some historical datasets, each with their own peculiarities, and then adding in some projection or prediction. Because data is essential, it is also a common stumbling block as business users consider beginning a new analytics and planning project.
Clients have expressed any number of data concerns to me over the years, but a handful of objections seem to float to the top most often. These concerns are typically logical but are actually not going to prevent them from implementing a new system or process. Analytics, regardless of the platforms being integrated, is about the details of the use case at hand. When viewed correctly through the lens of a use case, these data concerns almost always disappear. For this reason I think of these issues as "myths" that we have to dispel in order to advance and initiative.
The list below details the most common myths and why they won't stop you from successful execution on an analytics and planning project.
Myth #1: We don't have enough data.
How much is enough? That depends on the use case being pursued. Over hundreds and hundreds of past projects, only in two cases has a client actually not had enough data to achieve their goals. In both of those situations, the needed data had only just started being collected so the issue truly was low volume. Unless you've just started collecting data, the organization has enough historical data to begin working. Forecasting models can be more accurate with more data (to a point!) but a shorter time horizon shouldn't hold you back from making progress. Start now and get better over time as data accrues.
Myth #2: We have too much data.
This isn't a problem, this is a blessing! Certain use cases such as price optimization and demand planning work better with more historical data. Usually when a client raises this concern, the culprit is the level of detail. For a retailer or distributor, the base level data is by product by day and by customer or store. Across a whole company and large product catalog, that can be an intimidating amount of data. However, today's planning and analytics systems can scale to accommodate the required data. The key is determining what the required data is-- and often users only need a summarized level of data.
Myth #3: Our data warehouse has everything we need.
This just simply is never true. Structured data warehouses are great assets but a planning or analytics project typically needs data in a different format or needs to include data not in the warehouse married up to data from the warehouse. Its never as simple as just pulling the data in, but certain data like base-level transactions or historical sales data can usually be sourced from a warehouse easily.
Myth #4: We can never get away from Excel.
Even the largest companies' processes include a spreadsheet somewhere along the way. It seems strange but I've found it to be consistently true. Why is that? There are several reasons, each of which can be addressed during a project implementation.
- User Adoption: End users love Excel and are hesitant to give it up for a purpose-built application for fear of losing control. Engage your stakeholders early in the process to garner their support. There needs to be a proactive plan to drive adoption of a new system, it doesn't just happen. Furthermore, most purpose-built planning applications integrate well with Excel, so you end up with the best of all worlds.
- Lack of Integration: Excel is frequently used to overcome the lack of integration. Maybe Excel is consolidating data from two separate systems or preparing the data before it is loaded. Either way, this is a shortcoming, not a feature, and should be changed.
- Need for Input or Overrides: This is another system shortcoming. If end users need to modify the forecast or prediction from your analytics then your system should support that. End users should not have to use Excel to complete their work because the system won't accommodate them,
Myth #5: We can't maintain this ourselves.
For the past decade much of enterprise software purchasing has been driven by those outside the domain of the IT department. Line of business users are in the driver's seat in system selection and implementation. Independence and the ability to manage the system is paramount, but when it comes to data and integration, business users get cold feet. There's a fear of the unknown and integration sounds really complex.
The reality is that maintaining system and data integrations doesn't have to be complex. Maintenance does need to be considered during the design and tool selection phase, however. Think "People, Process, and Technology" relative to maintenance to be sure you create something that is effective and sustainable.
Myth #6: Our data is dirty or poor quality.
The truth is everyone's data is dirty to some degree. Experience has taught us what matters and what doesn't. With today's tools it is easy to build pipelines and processes that standardize and aggregate data for quality and consistency. But there is no magic bullet-- project stakeholders often have to put in the time and effort to map codes across systems and manage hierarchies.
Don't use data as a roadblock for an integration or project. The challenges with existing data can be overcome with design, tools, and process. For more information, look for more from QueBIT on relational datahubs and register for our upcoming webinar - Busting Myths about Data Prep and Integration.
