Maintaining consistency in your data over time

Too often companies are seeking for short-term solutions without considering a long-term vision of their organization data management.

A long-term vision is critical to a successful data management so that you do not end up digging your own grave with no turning back (or a costly one).

The capability to improve the decision-making process by your organization through reliable data, highly depends on the overall quality of your data, how fast and easy it is to access.

The overall data quality of your design is affected by how data are collected and stored.

Have a long-term vision for your data management:
- Data will grow over time
- Data migration will occur at some point
- Ease the adoption process
- Keep it simple

Data will grow over time:

Anticipating the growth of data is critical to maintaining quality over time. Keeping a clean data archive over time is challenging especially if all the work is done manually.

The explosive growth of data comes with its share of common problems regarding data management:
- Broken references
- Missing references
- Identical file name
- Missing custom properties
- …

Data migration
On the long run, when the data amount is growing, there might be a need to implement a new system where data migration will be required.

Data migration comes with big challenges. The migration success highly depends on the quality of your data. When performing a migration it is extremely hard to estimate how much effort and resources will be required, too often, they are way under estimated.

Keep in mind that data migration is not a simple project. It involves high risk, and it is very challenging, and sadly success is not guaranteed the first time.

A migration project is like an iceberg; you only see 10% before the actual migration starts.

Why data migration is so hard:

• Poorly managed data over the years
• Poor knowledge of the legacy data
• Overestimating the quality of the legacy data

Adoption process
In a company, new people are coming, and old ones are leaving… One of the big challenges when coming to a new company is to understand and adopt its processes. This is very true when it comes to data management.

If you have chosen a data management model that is easy to understand, and most importantly to use, then you will greatly ease the overall learning process.
Also having a clear process will not only ease the adoption with the engineering team but throughout the entire company.

Keep it simple
Establishing a set of common parameters, that everyone in your organization follows is not as easy as it seems. One reason for that is that every actor in an organization has his own views on how data should be handled.

Define rules that are easy to understand, deploy, use and most importantly adopt.
Making data consistent through standardization and automation is one way to keep your processes simple. Using standardization allows you to implement a more elaborative data management system with higher data quality as the errors will drop.

Keep in mind that the more complex it gets, the harder it is to maintain. Complex processes can work on the short term, but this might lead to a big and costly mess in the future. An ounce of prevention is certainly worth a pound of cure.

Writer:
Francois Simon, Sales Manager

LinkedIn profile