AI Production Secrets – From Failed POC to Live System
Reading Time: 10 minutesI am coming back with my quality writing after long period of pause, my last post was about Federated data in analytics platforms and today I am focusing on the…
Why Agentic AI Matters: Amazing Innovations for Data Platform Efficiency
Reading Time: 11 minutesAgentic AI is the new focus in the evolving world of AI. Since agentic AI systems are independent agents that can reason, they are quite different compared to traditional AI…
AWS S3 Tables & Apache Iceberg: The Future of Data Storage
Reading Time: 5 minutesIntroduction At AWS re:Invent 2024, AWS introduced Amazon S3 Tables, a purpose-built solution for managing tabular data at scale, built on the Apache Iceberg standard. AWS S3 Tables represent a…
AI Data Engineering: As A Data Engineer, You Really Need to Learn AI!
Reading Time: 5 minutesWhile leading data platform Architecture at Hilti and 10+ years of extensive experience working with data, I’ve witnessed numerous technological shifts in our field. But none have been as transformative…
Beyond Traditional Data Engineering: Your Guide to Emerging Specializations
Reading Time: 5 minutesIntroduction In today’s data-driven world, the field of data engineering has evolved far beyond its initial scope. What was once a relatively straightforward role focused on building data pipelines has…
Advanced Data Strategies for Large Language Models: A Technical Deep Dive
Reading Time: 6 minutesEnhance LLMs with advanced data integration for powerful, context-aware applications! Introduction In the rapidly evolving field of artificial intelligence, Large Language Models (LLMs) have emerged as powerful tools for natural…
Normalization vs Denormalization: Optimizing Data Modeling Techniques
Reading Time: 4 minutesData modeling is a crucial step in designing and implementing effective data storage and retrieval systems. Two widely used techniques in data modeling are normalization and denormalization. Let’s explore these…
Build Trust and Reliability — Data Platforms
Reading Time: 5 minutesKPIs for Data Platforms: Maintaining Data Quality, Monitoring, Observability, FinOps, and GreenOps A robust data engineering platform is crucial for organizations to leverage the power of their data. This platform…
Why Smart Data Engineers Use LLMs? A Proven Productivity Guide
Reading Time: 4 minutesIncrease productivity of your data engineering team and optimize your delivery pace with the help of LLMs. The world of data is exploding, and data engineers are the wranglers wrangling…
Why Your Data Architecture Needs FAIR Principles: A Proven Framework for Success
Reading Time: 3 minutesWhy FAIR principles? The amount of data we generate continues to explode at an unprecedented rate. Estimates suggest that in 2023, a staggering 120 zettabytes of data were created, captured,…
