Evaluating Databricks Job clusters Cost Savings and Performance in ADF
4 min readNov 3, 2023
The motivation behind writing this article is to explore the potential cost savings associated with Databricks job clusters and to determine whether using them within Data Factory or in Databricks workflows is the better option. As we are aware, all-purpose clusters tend to be more expensive than job clusters. This cost difference arises not only because interactive clusters require a minimum of about 10 minutes to shut off, but also because all-purpose clusters are more than twice the price of a job cluster ($0.15/DBU vs $0.40/BDU).
