Designing and Implementing Data Science Solutions on Microsoft Azure — Part 1

You will learn how to use Azure Machine Learning to operate machine learning workloads in the cloud. You will build on your existing data science and machine learning knowledge and learn how to leverage cloud services to perform machine learning at scale.

You are familiar with Python, and have experience of training machine learning models using common frameworks such as:

• Scikit-Learn
• PyTorch
• TensorFlow

You will be able to:

• Create an Azure Machine Learning workspace, and manage compute, data, and coding environments for machine learning workloads
• Use Azure Machine Learning for “no-code” machine learning model training and deployment.
• Create and run experiments that log metrics and train machine learning models.
• Create and manage datastores and datasets, and use data in machine learning experiments.
• Create and manage compute resources, and use them to run machine learning experiments at scale in the cloud.
• Use Pipelines to orchestrate machine learning operations.
• Deploy predictive models as real-time or batch inference services, and consume them from client applications.
• Find the optimal model for your data by using hyperparameter tuning and automated machine learning.
• Apply principles and techniques that support responsible machine learning practices.
• Monitor usage and data drift for deployed models.

Module 1: Getting Started with Azure Machine Learning

You will learn how to provision an Azure Machine Learning workspace and use it to manage machine learning assets such as data, compute, model training code, logged metrics, and trained models. You will learn how to use the web-based Azure Machine Learning studio interface as well as the Azure Machine Learning SDK and developer tools like Visual Studio Code and Jupyter Notebooks to work with the assets in your workspace.

Module 2: Visual Tools for Machine Learning

This module introduces the “Automated Machine Learning” and “Designer” visual tools, which You can use to train, evaluate, and deploy machine learning models without writing any code.

Module 3: Running Experiments and Training Models

You will get started with experiments that encapsulate data processing and model training code, and use them to train machine learning models.

Module 4: Working with Data

Data is a fundamental element in any machine learning workload, so in this module, you will learn how to create and manage datastores and datasets in an Azure Machine Learning workspace, and how to use them in model training experiments.

Module 5: Working with Compute

You’ll learn how to manage experiment environments that ensure consistent runtime consistency for experiments, and how to create and use compute targets for experiment runs.

Module 6: Orchestrating Machine Learning Workflows

Now that you understand the basics of running workloads as experiments that leverage data assets and compute resources, it’s time to learn how to orchestrate these workloads as pipelines of connected steps. Pipelines are key to implementing an effective Machine Learning Operationalization (ML Ops) solution in Azure, so you’ll explore how to define and run them in this module.

Module 7: Deploying and Consuming Models

How to deploy models for real-time inferencing, and for batch inferencing.

Module 8: Training Optimal Models

How you can use the azure Machine Learning SDK to apply hyperparameter tuning and automated machine learning, and find the best model for your data.

Module 9: Responsible Machine Learning

You explore some considerations and techniques for applying responsible machine learning principles.

Module 10: Monitoring Models

After a model has been deployed, it’s important to understand how the model is being used in production, and to detect any degradation in its effectiveness due to data drift. This module describes techniques for monitoring models and their data.

What is Azure Machine Learning?

Azure Machine Learning is a platform for operating machine learning workloads in the cloud.

Image- I

Built on the Microsoft Azure cloud platform, Azure Machine Learning enables you to manage:

· Scalable on-demand compute for machine learning workloads.

· Data storage and connectivity to ingest data from a wide range sources.

· Machine learning workflow orchestration to automate model training, deployment, and management processes.

· Model registration and management, so you can track multiple versions of models and the data on which they were trained.

· Metrics and monitoring for training experiments, datasets, and published services.

· Model deployment for real-time and batch inferencing.

In Azure, there are many services for storing data, including Azure Storage, Azure Data Lake Store, Azure SQL Database, Azure Cosmos DB, and others. There are also services that you can use to build “big data” processing solutions that transfer and transform data, including Azure Data Factory and Apache Spark engines in Azure HDInsight and Azure Databricks.

Azure also provides a huge array of services you can use to deliver applications for Web, mobile, and IoT devices; including Azure App Service, Azure Functions, and Azure IoT Edge. Azure also offers services for container-based deployment through services like Azure Container Services and Azure Kubernetes Services.

There are three primary types of user that Azure Machine Learning supports in this process:

• Data Scientists, who use their knowledge of statistics and data analytics to conduct analytical experiments and train machine learning models. These users typically work in Python or R, and use frameworks such as Scikit-Learn, PyTorch, and TensorFlow to train machine learning models.
• “Citizen” Data Scientists and App Developers, who don’t primarily work in the field of statistical data analysis, but who need to train machine learning models to support applications. These users can take advantage of graphical tools that abstract the underlying complexity of model training.
• Software engineers and operators, who need to operationalize machine learning to support applications and services. Their tasks typically involve using scripts or automated DevOps processes to manage model retraining and deployment, as well as overall application monitoring and .

I will share with you the series of articles I have been preparing for a long time about Designing and Data Science Solutions on Microsoft Azure. We will talk about all the modules.

We gave a brief introduction to Module 1 above. In order not to bore you, we will continue from where we left off in Module 1 in the next series.

See you soon 🙂

Stay healthy :)

Best regards,
Erdem