AI Production Secrets – From Failed POC to Live System
I am coming back with my quality writing after long period of pause, my last post was about Federated data in analytics platforms and today I am focusing on the…
AWS S3 Tables & Apache Iceberg: The Future of Data Storage
Introduction At AWS re:Invent 2024, AWS introduced Amazon S3 Tables, a purpose-built solution for managing tabular data at scale, built on the Apache Iceberg standard. AWS S3 Tables represent a…
AI Data Engineering: As A Data Engineer, You Really Need to Learn AI!
While leading data platform Architecture at Hilti and 10+ years of extensive experience working with data, I’ve witnessed numerous technological shifts in our field. But none have been as transformative…
Build Trust and Reliability — Data Platforms
KPIs for Data Platforms: Maintaining Data Quality, Monitoring, Observability, FinOps, and GreenOps A robust data engineering platform is crucial for organizations to leverage the power of their data. This platform…
Why Modern DataOps Matters: A Proven Guide to Data Platform Excellence
The concepts of DevOps, MLOps, DataOps, and FinOps represent distinct yet interconnected disciplines within the IT and operations landscape. In the evolving landscape of IT and operations, organizations continually seek…
Why Smart Companies Choose GreenOps: A Proven Guide to Sustainable Success
While digitizing world, the environmental impact of data-driven technologies has come under increasing scrutiny. Data platforms, forming the backbone of this digital ecosystem, are pivotal in determining the sustainability of…
Essential Sprint Planning Secrets: Transform Your Data Project Estimations
Streamlining Project Management & Project Estimations: Simplify Your Data Platform’s Agile Process with T-Shirt Sizing. Discover a Practical Approach to Sprint Planning and Team Efficiency. T-shirt size project estimations are…
Un-managing data: Data storage are expensive!
Topic Introduction In today’s digital-driven business landscape, data acts as the lifeblood of organizations, fueling decisions, innovations, and growth. However, in the rush to become data-driven, many companies fall into…
8 critical principle you should never miss while designing “Data Contracts”
Introduction In the realm of modern data architectures, the concept of a “Data Mesh” has emerged as a paradigm shift, addressing the complexities inherent in handling vast and varied data…
Serverless Computing May Not Be the Sole Path Forward — considerations before you design
In the rapidly evolving landscape of technology, serverless computing has emerged as a popular paradigm, especially in the context of application development. Its promise of eliminating the need for server…
