Implementing Dataops: A Scalable Framework for Modern Data Warehousing
Guardado en:
| Publicado en: | ProQuest Dissertations and Theses (2025) |
|---|---|
| Autor principal: | |
| Publicado: |
ProQuest Dissertations & Theses
|
| Materias: | |
| Acceso en línea: | Citation/Abstract Full Text - PDF |
| Etiquetas: |
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| Resumen: | DataOps has been coined as a novel term that emerged as a synthesis of data management practices with software engineering concepts, such as DevOps and Agile, with the goal of improving data quality and governance in enterprises. The proliferation of scratch table use and transformation tools, such as dbt, has led to an exponential increase in the number of data models, which complicates standardization efforts and increases maintenance overhead. Although the market is saturated with various flavors of text-to-SQL engines that promote increased productivity and self-service use in organizations, there are limited tools available to optimize individual queries, enforce consistency, or enhance data observability within the existing ecosystem. The many flavors of SQL, the de facto lingua franca of data processing, add even more complexity, as the code cannot be handled as easily as in less ambiguous languages like Python or Java, which have out-of-the-box linting and refactoring tools available. This study examines the impact of DataOps on modern enterprises, providing a programmatic solution that streamlines data operations through automated code review. The proposed framework introduces centralized SQL governance, embedded validation workflows, and observability features that promote collaboration and reduce redundancy. It leverages Python-based modular checks and a CI/CD pipeline to enforce validation in accordance with organizational standards. This research aims to bridge existing gaps and provide a scalable framework for effective DataOps implementation in modern data warehouse environments. |
|---|---|
| ISBN: | 9798265474971 |
| Fuente: | ProQuest Dissertations & Theses Global |