As you may know, the demand for Data Engineers has quadrupled in the last 2 years. There are more jobs than candidates in the job market, especially for US work authorized candidates. I have seen companies offering up to $1500 in referral bonuses.
My goal in writing this post is to give a self-motivated beginner a “simpler” learning path to becoming a data engineer. I have outlined the most important foundational skills needed to break into a Data Engineer role (junior role at least). The data engineer journey or landscape is truly overwhelming and I can say confidently there is…
The intended audience of this article are IT enthusiasts and beginner-level data engineers interested in understanding some data engineering principles
In the previous post (read here), I explored the rationale behind ETL and talked about how we can achieve full data extraction from a source system to a target system. In this post, I would focus our attention on incremental data extraction and some common design patterns.
Purpose: The goal of this article is to give an introductory guide on some basics of ETL as it relates to Data Engineering.
One of the most important and often overlooked core facets of data engineering is the creation of ETL pipelines. With the popularity of AI and ML projects and the concentration of demand for data scientists. It is easy to deem ETL as an old-fashioned approach to modern data analytics solutions. I have seen so many training programs overlook or give little attention to this subject area when teaching students about data engineering. ETL or ELT is actually…
The motivation for writing this is to explain the major difference between SMP and MPP platforms. I will also explain their appropriate use cases, pros, and cons.
Symmetric Multi-Processor (SMP) Architecture
“Symmetric multiprocessing (SMP) involves a multiprocessor computer hardware and software architecture where two or more identical processors are connected to a single, shared main memory, have full access to all input and output devices, and are controlled by a single operating system instance that treats all processors equally, reserving none for special purposes” reference
Let’s start with some history on analytical databases
Relational databases (ex SQL Server…
The goal of this article to illustrate step-by-step how to set up an Azure Application Gateway for your web services using the Azure Portal.
Before we begin with this tutorial, I would like to level set us with some basic understanding of the various components we will be configuring. I think it is crucial to understand what we are doing before doing it. Here are some basic questions one should know
3. What are the real-world use cases for Application Gateway?
4. What is the difference between HTTP…
This article is targeted towards IT professionals with less than 2 years of experience. After a few years in the industry here are some tips I would give folks starting on this journey.
It is better to admit that you don’t know and educate yourself than to assume you know and mislead yourself or someone. Always be open to learning something new, doing your research, and learning on your own.
Personally, this year has been humbling as I have been involved in various projects that are not strictly in my data engineering domain. I am not ashamed to say that…
My goal in writing this is to help someone that is yet to start, started but frustrated, or close to achieving their goal in their data analytics journey, gain inspiration and hope from my story. I hope they get encouraged by my countless failures then eventual moments of success.
I graduated from City College of New York with a bachelor’s in electrical engineering class of 2014. In my major, I took only 2 programming classes, but they were my favorite classes in my undergraduate degree. The first was Intro to programming in C/C++, the other was an advanced class in…
I am passionate about empowering and encouraging people of color in the data analytics career path