PinnedPublished inGeek CultureMy Guide to Becoming a “Market-Ready” Data Engineer With a $100 Investment in 3 Months or Less —…As you may know, the demand for Data Engineers has quadrupled in the last 2 years. There are more jobs than candidates in the job market…Jan 25, 20213Jan 25, 20213
Designing Cost-Optimized Data Pipelines esp. SparkI was once asked in an interview what Spark cluster configuration I would use to ingest a 100GB file into a data lake. My clarifying…1d ago1d ago
Table-Driven Design in Data Engineering: Making Your Code DynamicOne of the most powerful yet often overlooked techniques in data engineering is table-driven design. This approach involves creating…3d ago3d ago
Mastering Data Modeling: A Guide for Data EngineersCreating an exceptional data model for analytical workloads is both an art and a science. A well-designed data model drives faster…4d ago4d ago
Choosing the Right Database: Row vs. Columnar ExplainedAs data engineers, one of the most critical decisions we make is choosing the right database for our workloads. Picking between a row-based…5d ago5d ago
The Power of Respectful Disagreement in Tech: Why Questioning Your Tech Lead Can Drive Better…I have a confession to make. Throughout my career, I’ve often found myself butting heads with my tech leads or architects — especially…Mar 14Mar 14
Why Databricks Unity Catalog Managed Tables Are Better Than External TablesMy Journey with Unity Catalog Managed TablesMar 12Mar 12
Choosing the Right Data Storage Platform in Data EngineeringOne of the most critical decisions in data engineering is selecting the right data storage platform. With so many options available, it’s…Mar 10Mar 10
🚀 Is Apache Spark Really Dying? Let’s TalkThe world of data engineering moves fast. Every few months, a new tool emerges, claiming to be the next big thing.Mar 71Mar 71