PinnedPublished inGeek CultureMy Guide to Becoming a “Market-Ready” Data Engineer With a $100 Investment in 3 Months or Less —…As you may know, the demand for Data Engineers has quadrupled in the last 2 years. There are more jobs than candidates in the job market…Jan 25, 20213Jan 25, 20213
Create a Date Dimension in Databricks SQL with the SEQUENCE Function! 📆Building a Date Dimension is a cornerstone of any data warehouse, but it can often feel like a tedious task. Luckily, Databricks SQL makes…8h ago8h ago
Tackling Missing Data in Time SeriesWorking with time series data can be challenging, especially when gaps appear due to issues such as network interruptions or faulty devices…8h ago8h ago
Databricks Managed Tables vs External Tables Pros and ConsAdvantages of Databricks Managed Tables:Sep 27, 20241Sep 27, 20241
Implementing Slowly Changing Dimension (SCD) Type 2 for specific fields while performing Type 1…When working with Slowly Changing Dimensions (SCD), implementing Type 2 can be crucial to track historical data changes over time. This…Sep 15, 2024Sep 15, 2024
Databricks Workflow — Why, When and TipsDatabricks Workflows orchestrate data processing, machine learning, and analytics pipelines on the Databricks Data Intelligence Platform…Jul 6, 2024Jul 6, 2024
Customer Obsession vs. Competitor Obsession: A Call to Action for Data Analytics and AI CompaniesThere are few quotes as impactful as Jeff Bezos’ revelation that the secret to Amazon’s success lies in their customer obsession. This…Jun 21, 2024Jun 21, 2024
Optimizing Merge Statements in Databricks: Strategies for EfficiencyIf your Merge statement is taking as much time or more to complete compared to a full table rewrite, it indicates an optimization issue. In…Jun 16, 2024Jun 16, 2024
Cost Optimization for Databricks Clusters: A Data Engineer’s ApproachAs a data engineer, finding ways to save costs for your employer while using Databricks is essential. Here’s my approach, focusing on…May 25, 2024May 25, 2024