Hey, Cloud Gurus.
Welcome back to our series on
Meeting the Tools of the Trade.
In this lesson, we're continuing on
with Azure Synapse Pipelines.
These are a component of Azure Synapse Analytics,
and we're going to start off with a brief comparison
between them and Azure Data Factory.
We'll then jump over to a demo
and wrap everything up with a review.
So how do Azure Synapse Pipelines and Data Factory differ?
Well, not a lot.
It's kind of like looking in a mirror.
Azure Synapse Pipelines are built on the same technology
as Azure Data Factory.
There are some differences.
ADF has the SSIS activity.
It also has the Power Query activity.
Synapse has the ability to monitor Spark jobs for data flow,
but ADF has Azure Monitor Integration.
And this is just a small subset of the differences.
For the full list, you can follow this link.
I'll also include that link in the resources for the lesson,
but which tool you use basically comes down to,
what do you want to use it for?
For Synapse Pipelines,
we use it mostly for analytics projects.
When building an analytics solution,
Synapse Analytics is a one-stop shop
with a fully-integrated, design experience.
So you can look at Azure Synapse Analytics
as an all-in-one data and analytics solution
of which pipelines is just a part.
For those mostly wanting to do ETL and migration work,
you might lean toward Azure Data Factory instead.
With that background, let's jump into the Azure portal
and take a look at the action.
Here we are in the Azure portal,
within my Azure Synapse Analytics workspace.
I've opened up Azure Synapse Studio,
and already you can tell it has very much
the same look and feel as Azure Data Factory.
We do have some separate sections on the side here.
Data, Develop, and Integrate is where we want to go
to work with our Pipelines.
Within here, again, things look very much the same.
We can create our Pipeline
and we have many of the same activities as we did before.
There are some in here
that you don't find in Azure Data Factory
under our Synapse category,
but we have our same Copy data,
Data flow, HDInsight activities.
And so again, very much the same experience
as what you saw before.
As the 2 products develop, Azure Synapse
has some features that ADF does not,
but ADF has some features that Synapse Pipelines do not.
But the essence of how you move
and transform data with them is very much the same.
We're still using various activities
running on Databricks or Spark clusters in the background.
By way of review, Azure Synapse Pipelines
are built on the same technology as Azure Data Factory.
It does not share all ADF features.
Which tool you use will be determined
by your overall transformation and solution goals.
Thanks for joining me for this quick look
at Azure Synapse Pipelines.
When you're ready,
we'll jump into the next tool of the trade.
I'll see you there.