Many of you have already had the opportunity to use Azure Functions. Azure Functions is an on-demand cloud service that provides all the infrastructure and continuously updated resources needed to run your applications. You focus on the pieces of code that matter most to you, and Functions handles the rest. Functions provides serverless computing for Azure. You can use Functions to create web APIs, respond to database changes, process IoT streams, manage message queues, and more.
Besides being very efficient for real-time processing, when the functionality is limited to some basic processing, one may imagine some advanced scenarios of applying serverless architecture involving the Artificial Intelligence libraries such as Tensorflow or PyTorch. However, there might be some underlying difficulties in setting your solution up and running. The author of this blog has recently faced some of these challenges (still a rookie developer), so today, let's discuss some ways to cope with these problems.
Method 1: use wheels
I used to deploy Azure Functions using VSCode's Azure Functions extension, whish allows to simply send the command to execution, without having to deal with all the tricky configuration.
So, when I needed to install PyTorch on the Azure Function environment, I was pretty sure that it would take me a couple of minutes. For the first test, my init.py was quite simple and involved just few but very important lines of code:
YOLODIR = 'yolo'
yolomodelversion = 'yolov5l'
model = torch.hub.load('ultralytics/yolov5', yolomodelversion,
pretrained=True) # We can use yolov5n, yolov5s...
model.conf = 0.5 # Confidence threshold (0-1)
model.iou = 0.45 # NMS IoU threshold (0-1)
model.classes = [0, 2] # Persons & Cars only !!!So, I created the virtual environment, installed the packages (torch and torchvision), executed locally - it all worked like a charm. Then I opened the command palette using Ctrl + Shift + P and chose "deploy to function project".
And this is where the real challenge begun, because the deployment failed with the following message:
Generating summary of Oryx build
Deployment Log file does not exist in /tmp/oryx-build.log
The logfile at /tmp/oryx-build.log is empty. Unable to fetch the summary of build
Deployment Failed.The message is not this clear, is it?
So after a bit of research I've found out the following workaround. In two words, the suggestion was to simply change the dependency to:
torch==TORCH_VERSION+cpuYeah, seems pretty simple, but it is not. What I first tried to do, is, as Stackoverflow suggested, add cpu to the package name. Not surprisingly, it didn't work out, so I've decided to dive deeper in the subject and what I've found out:
Looks like this issue is related to virtual environment. Did you try recommended installation line in another/new one virtual environment? If it doesn't help the possible solution might be installing package using direct link to PyTorch and TorchVision builds for your system
So instead of using pypi, the guys on the internet suggested using wheels. But how can I use wheels in Azure Functions environment?
Well, finally it turned out to be quite straightforward. As the cpu wheels were not available on the pypi, I've looked through a couple of links proposed by Google after typing "torch cpu wheels" and I've discovered that the wheels were available on the torch's official website. So here are the steps I followed:
Then look for packages starting from cpu/
Select the last version (by now it's 1.11)
Now you need to choose the suitable version. If you have python 3.9 installed, so it should be cp 39 and so on and so forth. And then select between windows, linux or MacOS (N.B. please note that currently only Python 3.7, 3.8 and 3.9 are supported on Azure Function stack)
Copy the name of the package and add http://download.pytorch.org/whl/ at the beginning.
Repeat the same procedure for torchvision
It was a big surprise to me, but you can actually add the wheel url in requirements.txt along with classical packages. So me requirements file looked like this:
opencv-python
http://download.pytorch.org/whl/cpu/torch-1.11.0%2Bcpu-cp39-cp39-win_amd64.whl
http://download.pytorch.org/whl/cpu/torchvision-0.12.0%2Bcpu-cp39-cp39-win_amd64.whl
Pillowetc.
Works like a charm! But after that, I've found even more convenient way to install all the needed libraries.
Method 2: Azure Pipelines
Azure Pipelines allow you to Implement continuous integration and continuous delivery (CI/CD) for the app and platform of your choice. It is more sophisticated and more elegant way to deploy your function, but nevertheless, there are some prerequisites, like:
A GitHub account, where you can create a repository
An Azure DevOps organization
An ability to run pipelines on Microsoft-hosted agents
Once you've pushed the code to your github repository, do the following:
Sign in to your Azure DevOps organization and navigate to your project.
In your project, navigate to the Pipelines page. Then choose the action to create a new pipeline.
Walk through the steps of the wizard by first selecting GitHub as the location of your source code.
You might be redirected to GitHub to sign in. If so, enter your GitHub credentials.
When the list of repositories appears, select your sample app repository.
Azure Pipelines will analyze your repository and recommend a template. Select Save and run, then select Commit directly to the main branch, and then choose Save and run again.
A new run is started. Wait for the run to finish.
An example of YAML:
pool:
vmImage: ubuntu-latest
steps:
- task: UsePythonVersion@0
displayName: "Setting Python version to 3.7 as required by functions"
inputs:
versionSpec: '3.7'
architecture: 'x64'
- bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --output ./bin
fi
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
- task: ArchiveFiles@2
displayName: "Archive files"
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)"
includeRootFolder: false
archiveFile: "$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip"
- task: PublishBuildArtifacts@1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/build$(Build.BuildId).zip'
artifactName: 'drop'Please note, that your versionSpec may very depending on the stack you've chosen while creating your function app.
Once done you can deploy with the Azure Function App Deploy task. This task requires an Azure service connection as an input as it stores the credentials to connect from Azure Pipelines to Azure.
trigger:
- main
variables:
# Azure service connection established during pipeline creation
azureSubscription: <Name of your Azure subscription>
appName: <Name of the function app>
# Agent VM image name
vmImageName: 'ubuntu-latest'
- task: AzureFunctionApp@1 # Add this at the end of your file
inputs:
azureSubscription: <Azure service connection>
appType: functionAppLinux # default is functionApp
appName: $(appName)
package: $(System.ArtifactsDirectory)/**/*.zip
#Uncomment the next lines to deploy to a deployment slot
#Note that deployment slots is not supported for Linux Dynamic SKU
#deployToSlotOrASE: true
#resourceGroupName: '<Resource Group Name>'
#slotName: '<Slot name>'And the most important thing is that you don't need to provide the wheel. The real magic is that the agent builds a zip from your source code and deploys it to Azure directly.
So your requirements.txt now looks like:
opencv-python
torch
torchvision
PillowIsn't it pretty? And for the next update just push the changes, commit them and it will trigger the build job.
In today's short post we've covered some challenges and possible workarounds to set your ML-based Azure Function up and running. I didn't thoroughly investigate the issue and am still not quite sure why it works, but hopefully these small tips will save you a couple of hours, as I struggled a bit while working on this project. Maybe some other more convenient workarounds will be found. I will keep you updated.
Thank you for reading and may the force be with you!
Comments