Designing a Purpose-driven Data Analytics Architecture

Many organizations aspire to streamline their data analytics architecture. Many look to the latest technological solutions that may have the magic formula to solve all their data challenges. However, I have come across few organizations that create a purpose-driven data analytics architecture. What I mean by purpose-driven is a data architecture that is designed to suit the specific organization data needs and is well suited to meet all their unique business needs. This is very different from implementing a solution provided in Microsoft reference data architecture documentation like the below.

Source: Azure Modern Data Warehouse

Characteristics of a Poor Data Analytics Architecture

1. Poor data quality: If my data is sourced from the wrong source, you can guess the quality of data.
2. High Data Duplicity: If I store different versions of the same reference data in my warehouse, you can imagine the chaos it creates for the downstream processes.
3. High Latency between Source System and Downstream analytical system: If multiple ELT processes are hitting the same source system(relational DB not optimized for scale and analytics) for the same data, you can imagine how slow the data ingestion process will be.
4. High cost of new data analytical initiatives: considering all the above points, clearly…