PinnedPublished inGeek CultureMy Guide to Becoming a “Market-Ready” Data Engineer With a $100 Investment in 3 Months or Less —…As you may know, the demand for Data Engineers has quadrupled in the last 2 years. There are more jobs than candidates in the job market…Jan 25, 20213Jan 25, 20213
How Databricks’ Schema Evolution Saved My ETL Pipeline (And My Sanity)As a data engineer, I’ve encountered my fair share of frustrating ETL failures. You know the kind — the pipeline runs fine for weeks until…18h ago18h ago
How to Tell Your Story at Work Without Sounding Self-PromotionalTelling your story at work is essential for recognition, career growth, and building influence. But if you’re someone who prefers to “let…3d ago3d ago
🚀 Unlocking Uber’s Live Data Stack: Engineering Real-Time ExcellenceReal-time data processing isn’t just a feature — it’s the backbone of seamless user experiences. Ever wondered how Uber manages to process…Mar 25Mar 25
Designing Cost-Optimized Data Pipelines esp. SparkI was once asked in an interview what Spark cluster configuration I would use to ingest a 100GB file into a data lake. My clarifying…Mar 21Mar 21
Table-Driven Design in Data Engineering: Making Your Code DynamicOne of the most powerful yet often overlooked techniques in data engineering is table-driven design. This approach involves creating…Mar 191Mar 191
Mastering Data Modeling: A Guide for Data EngineersCreating an exceptional data model for analytical workloads is both an art and a science. A well-designed data model drives faster…Mar 19Mar 19
Choosing the Right Database: Row vs. Columnar ExplainedAs data engineers, one of the most critical decisions we make is choosing the right database for our workloads. Picking between a row-based…Mar 17Mar 17