Creating a Master Data Management (MDM) or a Data Warehouse can be a complex, lengthy and challenging process. A subtle wrong turn can change the compass of the roads ahead. You need to be careful about what you are trying to accomplish and what is actually required. It is very easy to get carried away. A clear clarification between must have, should have and could have demands is a must.
There are two schools of thought to build your MDM or Data Warehouse application:
- The Big Bang Theory:The MDM and the Data Warehouse application are built with all of the possible data items from the legacy system without relying on the business requirements. It is a Source Centric design.
- TheIncremental Approach – The MDM and the Data Warehouse application are built with only those data items that are described in the business requirements. It is a Target Centric design.
We have learned from our experience that although there is no such thing as a perfect methodology, the Incremental Approach with the Target Centric designis the most provenmethodology to build your MDM or Data Warehousing application.
We recommend to start with a the Incremental Approach and design the MDM or Data Warehouse with a 80:20 rule where eighty percent of the data items arestrictly based on the current business requirements, and the remaining twenty percent are based on the future use. This way we meet all the current business requirements and avoid being too rigid for not anticipating the future.
Why not bring everything?
We have been asked on several occasions – why not dump everything from the legacy systems? Our response to this question follows an analogy. Consider the MDM or data warehouse as your kitchen and legacy system as the grocery store. Will it make sense to go to the grocery store and buy everything? No. You make a wish-list of the items and buy only those items from the grocery store. You may buy some additional items, but certainly not the entire store! And this is exactly what should be done while building an MDM or Data Warehouse.