DEV Community: jaymin93

Plant Health evaluation using Raspberry Pi, Xamarin Forms App,Azure Function, Table,Storage, Key vault, Custom vision, Drone

jaymin93 — Mon, 31 Oct 2022 13:07:18 +0000

- Problem Statement

To Maintain healthiness of the Plant in larger area of farm is really challenging, Farmers needs to keep an eye on the plants they grow whether it is not affected by insects or disease, it requires manual inspection of plants by multiple farm workers and navigate through entire area of farm and inspecting them manually which is taking down many days for the farm workers and this needs to be repeated periodically which is consuming lots of time.

- Solution/ Architecture

This problem can be solved using Azure based solution where drone can capture images of elephant ear plant and identify if plant is affected by Insect or diseases using Azure Storage account which will hold the image captured by drone and it will be processed against the custom vision in which has trained images of healthy, and insect or diseases affected plants

Please refer Application flow

we will need following.

Azure Subscription: https://azure.microsoft.com/en-us/free/

.Net sdk 6.0 : - https://dotnet.microsoft.com/en-us/download/dotnet/6.0

Visual Studio 2022: - https://visualstudio.microsoft.com/vs/community/

Visual Studio needs following workloads.

Azure Custom Vision: -https://www.customvision.ai/

Click on create Project. Upload healthy and infected plant images and tag the uploaded images and perform the quick training.

Goto Performance Tab follow the steps and copy the url and key which will be used to access it.

Drone with Camera and GPS Module:- Drone supported by Parrot SDK and having FTP Server , learn more here https://www.parrot.com/

Please find the Github Link for the Project, you will need to enter correct Azure resources (e.g. storage , table , custom vision end point , ftp) values to run the Project.

https://github.com/jaymin93/PlantHealthApp

-Technical Details and Implementation of solution

1. PlantHealthConsoleApp :- .net 6 Console App

App will monitor Drone created FTP which will have images captured by Drone from farm, app will periodically check FTP server and images will be copied to destination directory from where it will be uploaded to Azure Storage as a Blob

Start Visual studio 2022 follow below steps

Please provide the Name and Path for the Project.

Select .net 6 LTS (For Production workloads LTS is highly recommended) Click on Create it will create Console Project.

Once Project has created, we will need below Nuget Packages which can be added from the below option

It will connect to Ftp server created by drone and get the values from the appsettings.json e.g. blob storage uri , ftp information and Secret Identifier

We will need Storage account and Azure keyvault Secret information for that go to https://portal.azure.com/

Create new Storage account

enter resource group name or select existing and add storage account name

select review+create.

Go to storage account you have created and select access key tab as below image

Create Container

KeyVault

Please enter correct resource group name and keyvault name then click on review + create

Navigate to keyvault and select Secret then Click on Generate Import

Please install nuget from the above menu by searching with the name.

Azure.Storage.Blobs
Microsoft.Azure.KeyVault
Microsoft.Azure.KeyVault.Models
Microsoft.Extensions.Configuration
Microsoft.Extensions.Hosting
WinSCP

Program.cs


using Azure.Storage.Blobs;
using Microsoft.Azure.KeyVault;
using Microsoft.Azure.KeyVault.Models;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using WinSCP;

namespace PlantHealthConsoleApp
{
    public class Program
    {
        private static IConfiguration? config;
        private static string? connectionstring;
        private static string? storageaccounturi;
        private static string? containername;
        private static string secretIdentifier = string.Empty;
        private static string imageDirPath = string.Empty;
        private static SecretBundle? secret;
        private static KeyVaultClient? client;
        private static System.Timers.Timer? timer;
        private static string imageProcessPath = string.Empty;

        public async static Task Main(string[] args)
        {
            HostBuilder builder = new HostBuilder();

            config = new ConfigurationBuilder()
             .AddJsonFile("appsettings.json", true, true)
             .Build();
            CheckForNewFileAdditionToDirectory();
            InitTimer();

            await builder.RunConsoleAsync();
        }
        private static void InitTimer()
        {
            timer ??= new System.Timers.Timer();
            timer.Enabled = true;
            timer.Interval = 60000;
            timer.Elapsed += Timermr_Elapsed;
        }

        private static void Timermr_Elapsed(object? sender, System.Timers.ElapsedEventArgs e)
        {
            GetFilesFromDronFTPServer(GetvaluesFromConfig("droneFtpUrl"), GetvaluesFromConfig("ftpUsername"), GetvaluesFromConfig("ftpPassword"), Convert.ToInt32(GetvaluesFromConfig("ftpport")));
        }

        private static void CheckForNewFileAdditionToDirectory()
        {
            imageDirPath = GetvaluesFromConfig("imageDirPath");
            FileSystemWatcher watcher = new()
            {
                Path = GetDirectoryForImageUpload()
            };
            watcher.Created += FileSystemWatcher_FileCreatedEvent;
            watcher.EnableRaisingEvents = true;
        }

        private static string GetDirectoryForImageUpload()
        {
            imageProcessPath = $"{Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments), imageDirPath)}";
            Console.WriteLine($"path is {imageProcessPath}");
            CreateDirectoryIfNotExist(imageProcessPath);
            return imageProcessPath;
        }

        private static void CreateDirectoryIfNotExist(string DirectoryPath)
        {
            if (!Directory.Exists(DirectoryPath))
            {
                Directory.CreateDirectory(DirectoryPath);
            }
        }

        private static string GetvaluesFromConfig(string key)
        {
            if (!string.IsNullOrEmpty(key) && config is not null)
            {
                return config[key];
            }
            return string.Empty;
        }

        private static void SetClientIDAndSecret()
        {
            TokenHelper.clientID ??= GetvaluesFromConfig("clientID");
            TokenHelper.clientSecret ??= GetvaluesFromConfig("clientSecret");
        }
        private async static void FileSystemWatcher_FileCreatedEvent(object sender, FileSystemEventArgs fileSystemEvent)
        {
            using (FileStream fileStream = new(fileSystemEvent.FullPath, FileMode.Open))
            {
                try
                {
                    storageaccounturi = GetvaluesFromConfig("storageaccounturi");
                    containername = GetvaluesFromConfig("containername");
                    secretIdentifier = GetvaluesFromConfig("secretIdentifier");
                    SetClientIDAndSecret();
                    client ??= new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(TokenHelper.GetAccessTokenAsync));
                    secret ??= await client.GetSecretAsync(secretIdentifier);
                    connectionstring ??= secret.Value;
                    if (!string.IsNullOrEmpty(fileSystemEvent.Name))
                    {
                        await UploadFileToAzureStorageAsync(connectionstring, fileSystemEvent.Name, containername, fileStream);
                    }
                }
                catch (Exception ex)
                {
                    Console.WriteLine(ex);
                }
            }
        }

        private static async Task<bool> UploadFileToAzureStorageAsync(string connectionString, string fileName, string containerName, Stream fileStream)
        {
            BlobClient blobClient = new BlobClient(connectionString, containerName, fileName);
            await blobClient.UploadAsync(fileStream);
            Console.WriteLine($"file {fileName} uploaded successfully");
            return await Task.FromResult(true);
        }

        private static void GetFilesFromDronFTPServer(string droneFtpUrl, string ftpUsername, string ftpPassword, int ftpport)
        {
            try
            {
                imageProcessPath ??= GetDirectoryForImageUpload();
                SessionOptions sessionOptions = new SessionOptions
                {
                    Protocol = Protocol.Ftp,
                    HostName = droneFtpUrl,
                    UserName = ftpUsername,
                    Password = ftpPassword,
                    PortNumber = ftpport
                };
                using (Session session = new Session())
                {
                    string droneCapturedImagePath = "/home/prt85463/images";
                    session.Open(sessionOptions);
                    session.GetFiles(droneCapturedImagePath, imageProcessPath).Check();
                    session.RemoveFiles(droneCapturedImagePath);
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex);
            }
        }
    }
}

Add new class by selecting from Solution explorer.

This class will be used to Get AccessToken to connect to keyvault and Get secret values from it.

ToeknHelper.cs


using Microsoft.IdentityModel.Clients.ActiveDirectory;
using System.Diagnostics;

namespace PlantHealthConsoleApp
{
    public static class TokenHelper
    {
        public static string clientID;
        public static string clientSecret;
        public static async Task<string> GetAccessTokenAsync(string authority, string resource, string scope)
        {
            var context = new AuthenticationContext(authority);
            ClientCredential credential = new ClientCredential(clientID,clientSecret);
            AuthenticationResult result = await context.AcquireTokenAsync(resource, credential);
            Trace.WriteLine(result.AccessToken);
            if (result == null)
                throw new InvalidOperationException("Failed to obtain the JWT token");

            return result.AccessToken;
        }
    }
}

Click on solution explorer and Select below Option, this app will be executing on Rasbpian Linux arm 64 so same configuration will be selected to Publish it.

From the option select Target runtime Linux-arm64 and Deployment as self contained ,learn more here https://learn.microsoft.com/en-us/dotnet/core/deploying/

Once Published copy executables to Rabpian and follow set permission chmod +x filename and then use ./filename to execute application, learn more from here https://learn.microsoft.com/en-us/dotnet/iot/deployment

2. PlantHealthApp :- Azure Function Blob Trigger with .net 6 as Target Framework.

Function will be triggered once there is new image uploaded by console app, which will be sent to Azure Custom Vision for prediction if plant is affected by insect or dieses, details of the affected plant will be stored in Azure Table

Select Azure Function

Provide Connection string name and click on finish.

Please install following nuget packages

Azure.Security.KeyVault.Secrets
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training
Microsoft.Azure.KeyVault
Microsoft.Azure.WebJobs.Extensions.Storage
Microsoft.Identity.Client
Microsoft.IdentityModel.Clients.ActiveDirectory
RestSharp

GetPlantHealth.cs


using System;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage.Table;
using Microsoft.WindowsAzure.Storage;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Net;
using Microsoft.Azure.KeyVault;

namespace GetPlantHealthDetails
{
    public class GetPlantHealth
    {
        private static CloudStorageAccount storageAccount = null;
        private static CloudTableClient tableClient = null;
        private static CloudTable table = null;
        private static KeyVaultClient client = null;
        private static Microsoft.Azure.KeyVault.Models.SecretBundle connectionstring = null;
        private static string tableName = "PlantHealthAppTable";
        private static string secretIdentifier = "https://planthealthappsecret.vault.azure.net/secrets/storageAccountConnectionString/92f4ed20ff4041ae8b05303f7baf79f7";

        [FunctionName("GetPlantHealth")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
            ILogger log)
        {
            SetClientIDAndSecret();
            client ??= new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(TokenHelper.GetAccessTokenAsync));
            connectionstring ??= await client.GetSecretAsync(secretIdentifier);
            storageAccount ??= CloudStorageAccount.Parse(connectionstring.Value);
            tableClient ??= storageAccount.CreateCloudTableClient();
            table ??= tableClient.GetTableReference(tableName);

            string rowkey = req.Query["RowKey"];
            if (string.IsNullOrEmpty(rowkey))
            {
                return new OkObjectResult(await GetPlantHealthDeatilsAsync(log));
            }
            else
            {
                return new OkObjectResult(await UpdatePlantHealthDeatilsByRowkeyAsync(rowkey, log));
            }
        }

        private static async Task<List<PlantHealthDeatils>> GetPlantHealthDeatilsAsync(ILogger logger)
        {
            try
            {
                List<PlantHealthDeatils> plantHealthDeatilsList = new List<PlantHealthDeatils>();
                TableQuery<PlantHealthDeatils> query;
                query = new TableQuery<PlantHealthDeatils>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, $"{tableName}"));
                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<PlantHealthDeatils> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    foreach (var entity in resultSegment.Results.OrderBy(x => x.Pesticidesprayed))
                    {
                        PlantHealthDeatils details = new PlantHealthDeatils
                        {
                            longitude = entity.longitude,
                            ImageURL = entity.ImageURL,
                            latitude = entity.latitude,
                            Pesticidesprayed = entity.Pesticidesprayed,
                            CapturedTime = entity.CapturedTime,
                            RowKey = entity.RowKey,
                            ETag = entity.ETag,
                            PartitionKey = entity.PartitionKey
                        };
                        plantHealthDeatilsList.Add(details);
                    }
                } while (token != null);
                return plantHealthDeatilsList;
            }
            catch (Exception exp)
            {
                logger.LogError(exp, "Unable to GetPlantHealthDeatils");
                return default;
            }
        }

        private static async Task<HttpResponseMessage> UpdatePlantHealthDeatilsByRowkeyAsync(string rowkey, ILogger logger)
        {
            try
            {
                PlantHealthDeatils plantHealthDeatilsList = new PlantHealthDeatils();
                TableQuery<PlantHealthDeatils> query;
                query = new TableQuery<PlantHealthDeatils>().Where(TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.Equal, $"{rowkey}"));
                TableContinuationToken token = null;

                TableQuerySegment<PlantHealthDeatils> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);

                var plantdetail = resultSegment.Results.FirstOrDefault();
                plantdetail.Pesticidesprayed = true;
                var operation = TableOperation.Replace(plantdetail);
                await table.ExecuteAsync(operation);

                return new HttpResponseMessage(HttpStatusCode.NoContent);
            }
            catch (Exception exp)
            {
                logger.LogError(exp, "Unable to Update PlantHealthDeatils");
                return default;
            }
        }

        private static string GetEnviromentValue(string key)
        {
            return Environment.GetEnvironmentVariable(key);
        }

        private static void SetClientIDAndSecret()
        {
            TokenHelper.clientID ??= GetEnviromentValue("clientID");
            TokenHelper.clientSecret ??= GetEnviromentValue("clientSecret");
        }
    }
}

Add new class file as below

PlantHealthDeatils.cs this class will be inherited by TableEntity class that will be used to communicate as model class for the Azure Table , please notice parameterised constructor has two important arguments PartitionKey , RowKey


using Microsoft.WindowsAzure.Storage.Table;
using System;

namespace GetPlantHealthDetails
{
    public class PlantHealthDeatils : TableEntity
    {
        public PlantHealthDeatils()
        {

        }
        public PlantHealthDeatils(string skey, string srow)
        {
            PartitionKey = skey;
            RowKey = srow;
        }
        public DateTime CapturedTime { get; set; }
        public string longitude { get; set; }
        public string latitude { get; set; }
        public string ImageURL { get; set; }
        public bool Pesticidesprayed { get; set; } = false;


    }
}

For Publishing to the Azure we will be creating Azure function

Click review + create

Once Azure function is created, we can deploy it to Azure

From Visual Studio select below

local.settings.json contais key value pair for stoarge account url,custom vision and Table, this needs to be added to Azure function from Portal

Once Publish is successful, we can upload image to container and Blob Trigger will be executed, affected plant deatils will be stored in the Azure Table

3. GetPlantHealthDetails :- Azure Function Http Trigger with .net 6 as Target Framework.

Function will retrieve data of the affected plant from Azure Table and will serve the response to the Xamarin Forms Based App running on Windows, Android, Ios

rowkey based query string can be sent to Azure Function which will update the Pesticide spray status flag of particular record which can be done from the agricultural drone spraying pesticide.

Add new Project of Azure function (please refer above screen shot),we will be adding Http Trigger Azure function Trigger Type.

Please install following nuget packages

Azure.Identity
Azure.Security.KeyVault.Secrets
Microsoft.Azure.Functions.Extensions
Microsoft.Azure.KeyVault
Microsoft.Identity.Client
Microsoft.IdentityModel.Clients.ActiveDirectory
System.Configuration.ConfigurationManager

GetPlantHealth.cs


using System;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage.Table;
using Microsoft.WindowsAzure.Storage;
using System.Collections.Generic;
using System.Linq;
using System.Net.Http;
using System.Net;
using Microsoft.Azure.KeyVault;

namespace GetPlantHealthDetails
{
    public class GetPlantHealth
    {
        private static CloudStorageAccount storageAccount = null;
        private static CloudTableClient tableClient = null;
        private static CloudTable table = null;
        private static KeyVaultClient client = null;
        private static Microsoft.Azure.KeyVault.Models.SecretBundle connectionstring = null;
        private static string tableName = "PlantHealthAppTable";
        private static string secretIdentifier = "https://planthealthappsecret.vault.azure.net/secrets/storageAccountConnectionString/92f4ed20ff4041ae8b05303f7baf79f7";

        [FunctionName("GetPlantHealth")]
        public static async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req,
            ILogger log)
        {
            SetClientIDAndSecret();
            client ??= new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(TokenHelper.GetAccessTokenAsync));
            connectionstring ??= await client.GetSecretAsync(secretIdentifier);
            storageAccount ??= CloudStorageAccount.Parse(connectionstring.Value);
            tableClient ??= storageAccount.CreateCloudTableClient();
            table ??= tableClient.GetTableReference(tableName);

            string rowkey = req.Query["RowKey"];
            if (string.IsNullOrEmpty(rowkey))
            {
                return new OkObjectResult(await GetPlantHealthDeatilsAsync(log));
            }
            else
            {
                return new OkObjectResult(await UpdatePlantHealthDeatilsByRowkeyAsync(rowkey, log));
            }
        }

        private static async Task<List<PlantHealthDeatils>> GetPlantHealthDeatilsAsync(ILogger logger)
        {
            try
            {
                List<PlantHealthDeatils> plantHealthDeatilsList = new List<PlantHealthDeatils>();
                TableQuery<PlantHealthDeatils> query;
                query = new TableQuery<PlantHealthDeatils>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, $"{tableName}"));
                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<PlantHealthDeatils> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    foreach (var entity in resultSegment.Results.OrderBy(x => x.Pesticidesprayed))
                    {
                        PlantHealthDeatils details = new PlantHealthDeatils
                        {
                            longitude = entity.longitude,
                            ImageURL = entity.ImageURL,
                            latitude = entity.latitude,
                            Pesticidesprayed = entity.Pesticidesprayed,
                            CapturedTime = entity.CapturedTime,
                            RowKey = entity.RowKey,
                            ETag = entity.ETag,
                            PartitionKey = entity.PartitionKey
                        };
                        plantHealthDeatilsList.Add(details);
                    }
                } while (token != null);
                return plantHealthDeatilsList;
            }
            catch (Exception exp)
            {
                logger.LogError(exp, "Unable to GetPlantHealthDeatils");
                return default;
            }
        }

        private static async Task<HttpResponseMessage> UpdatePlantHealthDeatilsByRowkeyAsync(string rowkey, ILogger logger)
        {
            try
            {
                PlantHealthDeatils plantHealthDeatilsList = new PlantHealthDeatils();
                TableQuery<PlantHealthDeatils> query;
                query = new TableQuery<PlantHealthDeatils>().Where(TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.Equal, $"{rowkey}"));
                TableContinuationToken token = null;

                TableQuerySegment<PlantHealthDeatils> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);

                var plantdetail = resultSegment.Results.FirstOrDefault();
                plantdetail.Pesticidesprayed = true;
                var operation = TableOperation.Replace(plantdetail);
                await table.ExecuteAsync(operation);

                return new HttpResponseMessage(HttpStatusCode.NoContent);
            }
            catch (Exception exp)
            {
                logger.LogError(exp, "Unable to Update PlantHealthDeatils");
                return default;
            }
        }

        private static string GetEnviromentValue(string key)
        {
            return Environment.GetEnvironmentVariable(key);
        }

        private static void SetClientIDAndSecret()
        {
            TokenHelper.clientID ??= GetEnviromentValue("clientID");
            TokenHelper.clientSecret ??= GetEnviromentValue("clientSecret");
        }
    }
}

PlantHealthDeatils.cs


using Microsoft.WindowsAzure.Storage.Table;
using System;

namespace GetPlantHealthDetails
{
    public class PlantHealthDeatils : TableEntity
    {
        public PlantHealthDeatils()
        {

        }
        public PlantHealthDeatils(string skey, string srow)
        {
            PartitionKey = skey;
            RowKey = srow;
        }
        public DateTime CapturedTime { get; set; }
        public string longitude { get; set; }
        public string latitude { get; set; }
        public string ImageURL { get; set; }
        public bool Pesticidesprayed { get; set; } = false;

    }
}

local.settings.json contais the information for the clientID,clientSecret,tableName,secretIdentifier , this needs to be added to Azure function from Portal

For publish please follow the above Function Publish steps to Azure.

4. PlantHealthAppXam :- Xamarin Forms based Uwp , Android , Ios app which will display information for the infected plants with images by querying the data to GetPlantHealthDetails ,as well as shows the Location on Map using the Longitude and Latitude Azure Function Http Trigger.

Use Add new Project(refer above screenshot) to solution and add Xamarin Forms Project

Give correct Name and location to it then click on Next select below option

Once Project is created you will see belows Projects added to solution

Goto 1 st Project from the above image in view folder we will be updating views as below.

Xamarin forms Project uses viewmodel and binding with MVVM , learn more from below link.

https://learn.microsoft.com/en-us/xamarin/xamarin-forms/enterprise-application-patterns/mvvm

ItemDetailPage.xaml


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="PlantHealthAppXam.Views.ItemDetailPage"
             Title="{Binding Title}">

    <StackLayout Spacing="20" Padding="15">
        <Image Source="{Binding IMGURL}"></Image>
        <Button Text="Open Map" Command="{Binding OpenMapCommand}"></Button>
    </StackLayout>
</ContentPage>

ItemDetailPage.xaml.cs


using PlantHealthAppXam.ViewModels;
using System.ComponentModel;
using System.Threading.Tasks;
using Xamarin.Essentials;
using Xamarin.Forms;

namespace PlantHealthAppXam.Views
{
    public partial class ItemDetailPage : ContentPage
    {
        public ItemDetailPage()
        {
            InitializeComponent();
            BindingContext = new ItemDetailViewModel();
        }
    }
}

ItemDetailViewModel.cs in ViewModels folder


using PlantHealthAppXam.Models;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Xamarin.Essentials;
using Xamarin.Forms;

namespace PlantHealthAppXam.ViewModels
{
    [QueryProperty(nameof(IMGURL), nameof(IMGURL))]
    [QueryProperty(nameof(longitude), nameof(longitude))]
    [QueryProperty(nameof(latitude), nameof(latitude))]
    public class ItemDetailViewModel : BaseViewModel
    {
        private string imgurl;
        public Command OpenMapCommand { get; }
        public string longitude { get; set; }
        public string latitude { get; set; }

        public string Id { get; set; }

        public string IMGURL
        {
            get => imgurl;
            set => SetProperty(ref imgurl, value);
        }


        public ItemDetailViewModel()
        {
            OpenMapCommand = new Command(async () => await OpenMapByLongitudeLatitude(longitude,latitude));
        }

        public async Task OpenMapByLongitudeLatitude(string Longitude, string Latitude)
        {
            var location = new Location(Convert.ToDouble(Longitude), Convert.ToDouble(Latitude));
            await Map.OpenAsync(location);
        }
    }
}

ItemsPage.xaml


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="PlantHealthAppXam.Views.ItemsPage"
             Title="{Binding Title}"
             xmlns:local="clr-namespace:PlantHealthAppXam.ViewModels"  
             xmlns:model="clr-namespace:PlantHealthAppXam.Models"  
             x:Name="BrowseItemsPage">

    <ContentPage.ToolbarItems>
        <!--<ToolbarItem Text="Add" Command="{Binding AddItemCommand}" />-->
    </ContentPage.ToolbarItems>
    <!--
      x:DataType enables compiled bindings for better performance and compile time validation of binding expressions.
      https://docs.microsoft.com/xamarin/xamarin-forms/app-fundamentals/data-binding/compiled-bindings
    -->
    <RefreshView x:DataType="local:ItemsViewModel" Command="{Binding LoadItemsCommand}" IsRefreshing="{Binding IsBusy, Mode=TwoWay}">
        <CollectionView x:Name="ItemsListView"
                ItemsSource="{Binding ItemsList}"
                SelectionMode="None">
            <CollectionView.ItemTemplate>
                <DataTemplate>
                    <StackLayout Padding="10" Orientation="Horizontal" x:DataType="model:PlantHealthDeatils">
                        <StackLayout Orientation="Vertical">
                            <StackLayout Orientation="Horizontal">
                                <Label FontAttributes="Bold" Text="Longitude :"></Label>
                                <Label Text="{Binding longitude}" 
                            LineBreakMode="NoWrap" 
                            Style="{DynamicResource ListItemDetailTextStyle}" 
                            FontSize="13" />
                            </StackLayout>
                            <StackLayout Orientation="Horizontal">
                                <Label FontAttributes="Bold" Text="Latitude :"></Label>
                                <Label Text="{Binding latitude}" 
                            LineBreakMode="WordWrap" 
                            Style="{DynamicResource ListItemDetailTextStyle}"
                            FontSize="13" />
                            </StackLayout>
                            <StackLayout Orientation="Horizontal">
                                <Label FontAttributes="Bold" Text="Captured Time :"></Label>
                                <Label Text="{Binding CapturedTime}" 
                            LineBreakMode="WordWrap" 
                            Style="{DynamicResource ListItemDetailTextStyle}"
                            FontSize="13" />
                            </StackLayout>
                            <StackLayout Orientation="Horizontal">
                                <Label  FontAttributes="Bold" Text="Pesticide Sprayed :" Margin="0,5,0,0"></Label>
                                <CheckBox IsEnabled="False" IsChecked="{Binding Pesticidesprayed}" ></CheckBox>
                            </StackLayout>
                        </StackLayout>
                        <Image Source="{Binding ImageURL}"  HorizontalOptions="EndAndExpand" HeightRequest="100" WidthRequest="100"></Image>
                        <StackLayout.GestureRecognizers>
                            <TapGestureRecognizer 
                                NumberOfTapsRequired="1"
                                Command="{Binding Source={RelativeSource AncestorType={x:Type local:ItemsViewModel}}, Path=ItemTapped}"     
                                CommandParameter="{Binding .}">
                            </TapGestureRecognizer>
                        </StackLayout.GestureRecognizers>
                    </StackLayout>
                </DataTemplate>
            </CollectionView.ItemTemplate>
        </CollectionView>
    </RefreshView>
</ContentPage>

ItemsViewModel.cs in ViewModels


using Newtonsoft.Json;
using PlantHealthAppXam.Models;
using PlantHealthAppXam.Views;
using RestSharp;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Net.Http;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace PlantHealthAppXam.ViewModels
{
    public class ItemsViewModel : BaseViewModel
    {
        private PlantHealthDeatils _selectedItem;

        public ObservableCollection<PlantHealthDeatils> ItemsList { get; }
        public Command LoadItemsCommand { get; }
        public Command<PlantHealthDeatils> ItemTapped { get; }

        public ItemsViewModel()
        {
            Title = "Plant List";
            ItemsList = new ObservableCollection<PlantHealthDeatils>();
            LoadItemsCommand = new Command(async () => await ExecuteLoadItemsCommand());

            ItemTapped = new Command<PlantHealthDeatils>(OnItemSelected);
        }

        async Task ExecuteLoadItemsCommand()
        {
            IsBusy = true;

            try
            {
                ItemsList.Clear();
                var items = await GetDataAsync().ConfigureAwait(false);
                foreach (var item in items)
                {
                    ItemsList.Add(item);
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex);
            }
            finally
            {
                IsBusy = false;
            }
        }

        public void OnAppearing()
        {
            IsBusy = true;
            SelectedItem = null;
        }

        public PlantHealthDeatils SelectedItem
        {
            get => _selectedItem;
            set
            {
                SetProperty(ref _selectedItem, value);
                OnItemSelected(value);
            }
        }

        async void OnItemSelected(PlantHealthDeatils item)
        {
            if (item == null)
                return;

            // This will push the ItemDetailPage onto the navigation stack
            //Shell.Current.GoToAsync($"//home/bottomtab2?name={"Cat"}&test={"Dog"}");
            await Shell.Current.GoToAsync($"{nameof(ItemDetailPage)}?{nameof(ItemDetailViewModel.IMGURL)}={item.ImageURL}&{nameof(ItemDetailViewModel.longitude)}={item.longitude}&{nameof(ItemDetailViewModel.latitude)}={item.latitude}");
        }

        public async Task<List<PlantHealthDeatils>> GetDataAsync()
        {
            var client = new RestClient("https://getplanthealthdetails.azurewebsites.net/api/GetPlantHealth?code=Ffcqj7PbO68QaTg2zWRNN7yp76kyYXNr8YBC_qw-jUXSAzFuAIrvKw==");
            var request = new RestRequest();
            request.Method = Method.Get;
            var response = await client.ExecuteAsync(request);
            return JsonConvert.DeserializeObject<List<PlantHealthDeatils>>(response.Content.ToString());
        }

    }
}

ItemsPage.xaml.cs


using PlantHealthAppXam.Models;
using PlantHealthAppXam.ViewModels;
using PlantHealthAppXam.Views;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Xamarin.Forms;
using Xamarin.Forms.Xaml;

namespace PlantHealthAppXam.Views
{
    public partial class ItemsPage : ContentPage
    {
        ItemsViewModel _viewModel;

        public ItemsPage()
        {
            InitializeComponent();

            BindingContext = _viewModel = new ItemsViewModel();
        }

        protected override void OnAppearing()
        {
            base.OnAppearing();
            _viewModel.OnAppearing();
        }
    }
}

PlantHealthDeatils.cs


using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Text;

namespace PlantHealthAppXam.Models
{
    public class PlantHealthDeatils 
    {
        public PlantHealthDeatils()
        {

        }

        public DateTime CapturedTime { get; set; }

        public string longitude { get; set; }

        public string latitude { get; set; }
        public string ImageURL { get; set; }
        public bool Pesticidesprayed { get; set; } = false;
        public string ETag { get; set; }
        public string RowKey { get; set; }
        public string PartitionKey { get; set; }

    }
}

App running on Windows 11, Android, Ios

- Challenges in implementing the solution

Raspbian (Linux distro) is case sensitive for file naming, I have used camel casing in file name AppSettings.json on windows it worked fine but on Linux it was null values so after debugging the app on Linux I came to know about it and used same casing later in file name to fix it.

- Business Benefit

Human efforts can be saved with the Azure based solution which will reduce the human efforts and increase revenue generation for farmers as farm workers can focus on another task. Using agriculture drones using longitude and latitude we can spray pesticide to selected area.

Protect Farm from Wild Animal using Azure cognitive services Custom Vision, Azure Functions, Table, .Net 5 , Xamarin App

jaymin93 — Sat, 03 Apr 2021 07:07:29 +0000

Problem statement

Farmers are facing real challenges with the Crop attacked by Wild Animals , Most of the times Monkeys are biggest threat or Problem for them since once they attack on farm they eat and damage crop and causes them huge loss of ready crop and fruits that cost them financially loss.

Even in some case many farmers has left farming and switch to other business due to frequent attack of monkeys and financially loss.

refer link below for same kind of issues farmer facing in real life

https://english.mathrubhumi.com/agriculture/agri-news/these-farmers-of-puralimala-valley-gave-up-farming--1.4645479

https://www.downtoearth.org.in/coverage/wildlife-biodiversity/out-of-control-why-monkeys-are-a-menace-50817

https://indianexpress.com/article/india/india-news-india/man-vs-monkey-how-farmers-are-grappling-with-losses-that-have-hit-himachals-economy/

This incident of monkey attacked farm happened to my relative and costed them huge financially loss due to Monkey's 25-30 group attack on his farm , I came to know via the conversation with him , he also told that they could have took the action and make them run away from farm if somebody was aware that monkey group has attacked to Farm .

Solution

So to solve out this real world Problem idea came into my mind as I was using many azure services already.
Azure cognitive services Custom Vision can help me here to identity the entrance of monkey from the captured image of farm and I can take preventive action against the entrance of Monkey by Playing Lion roar sound also alerting farmers for the incident which is effective solution and can help to protect farm.

👩‍🌾Protect Farmer 🐵 is application developed using the Azure cognitive services Custom Vision , Azure Functions with Blob Trigger, Azure Table , .Net 5 Console App, Xamarin App (UWP ,Android , Ios ) to display captured image of farm with presence of Monkey existence and take preventive action.

Here is the Link of working App , Just Enter correct storage account table , storage and custom vision resource value to run it

https://github.com/jaymin93/PtrotectFarm

Explanation of the concepts applied

Application Flow

Protect Farmer has 3 Solutions

1 .Net 5 Console App :- which will monitor for new captured images from CCTV or Surveillance camera , once it has new image captured it will upload to Azure Blob Storage

Also .net 5 Console App will help to play lion roar sound once it has retrieved the monkey detection

2 Azure Function Blob Trigger :- Azure Function with Blob Trigger , it will be triggered once new image uploaded to blob storage.

It will be sent to Azure cognitive services Custom vision
for identifying the existence of Monkey in the captured image

If Monkey existence found in captured image Azure Function will insert details to Azure Table along with Lion roar playing sound status updated to Azure Table

3 Xamarin App - UWP , Android , Ios :- Xamarin App display's Captured image that let's review farmer the Monkey existence with Result from Azure cognitive services Custom Vision stored in Azure Table also controls the Lion Roar Sound

Let's dive in to the code and how to build each solution.

what you need is

Visual Studio 2019 :- get visual studio 2019
https://visualstudio.microsoft.com/downloads/
there is community edition which is free.

Azure Account :- you can get azure account from here

https://azure.microsoft.com/en-us/free/

And there is 25 services which are always free , if you are student Microsoft offers 100 USD credits as detailed here
Get 100 USD Student Credit

https://azure.microsoft.com/en-us/free/students/

.Net 5 SDK :- get .Net 5 SDK Here https://dotnet.microsoft.com/download/dotnet/5.0

Xamarin and Azure Function workload for visual studio :- Please install workloads shown in red box as part of your visual studio 2019 installation

Building .NET 5 Console App

Start Visual Studio 2019, Click on Create New Project as shown in red box

click on start new project

Next screen will ask you for project name and directory please select directory and provide name PtrotectFarmByWildAnimal, once it has been created go to Program.cs and replace with following code

And Intsall following Nuget Packages as below

Nuget packages from tools -> Nuget package manager -> Manage Nuget for solution

Azure.Storage.Blobs;
Microsoft.Extensions.Configuration;
Microsoft.Extensions.Hosting;
Microsoft.WindowsAzure.Storage;
Microsoft.WindowsAzure.Storage.Table;

In App settings it requires connectionstring , storageaccounturi , containername which can be retrieved from the azure blob storage account .

To create Azure Storage Account go to https://portal.azure.com/

enter resource group name or select existing and add storage account name

click on networking , data protection , advance , tags keep default selection and click on review and create button , once storage account is created it will be displayed in storage account list .

Go to storage account you have created and select access key tab as below image

then click on show key and from there you can get connection string ,

go to storage explorer preview and create new blob container , notice container name and storage account uri and set correct values in appsetting.json

Program.cs


using Azure.Storage.Blobs;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using ProtectFarm.Models;
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using System.Timers;
using System.Linq;
using System.Linq.Expressions;
using System.Diagnostics;
using System.Media;
using System.Linq.Expressions;

namespace ProtectFarmers
{
    class Program
    {
        static IConfiguration config;

        internal static string TableName = "helpfarmer";

        const string status = "status";

        private const string playsound = "playing";

        private const string stopsound = "stopped";


        public static string CurrentSoundPlayingStatus
        {
            get; set;
        }

        public static string connectionstring { get; set; }

        public static string storageaccounturi { get; set; }

        public static string containername { get; set; }

        public static Timer tmr;

        public static SoundPlayer soundPlayer;

        public static object obj = new object();

        public async static Task Main(string[] args)
        {
            HostBuilder builder = new HostBuilder();

            config = new ConfigurationBuilder()
             .AddJsonFile("appsettings.json", true, true)
             .Build();

            FileSystemWatcher fileSystemWatcher = new FileSystemWatcher();

//set correct directory path as needed.
            fileSystemWatcher.Path = "G:\\cameradir";
            fileSystemWatcher.Created += FileSystemWatcher_Created;
            fileSystemWatcher.EnableRaisingEvents = true;
            await builder.RunConsoleAsync();
        }

        private async static void FileSystemWatcher_Created(object sender, FileSystemEventArgs e)
        {
            using (FileStream fileStream = new FileStream(e.FullPath, FileMode.Open))
            {
                connectionstring = config["connectionstring"];
                storageaccounturi = config["storageaccounturi"];
                containername = config["containername"];

                await UploadFileToAzureStorageAsync(fileStream, storageaccounturi, connectionstring, containername, DateTime.Now.ToString("dd-MM-yyyy-HH-mm-ss") + e.Name);
            }

            if (tmr == null)
            {
                tmr = new Timer();
                tmr.Enabled = true;
                tmr.Interval = 15000;
                tmr.Elapsed += Tmr_Elapsed;
            }
        }

        private async static void Tmr_Elapsed(object sender, ElapsedEventArgs e)
        {
            await GetCurrentSoundPlayingStatusAsync(connectionstring);


            if (CurrentSoundPlayingStatus == playsound)
            {
                lock (obj)
                {
                    PlayLionRoarSound(true);
                }
            }
            else
            {
                lock (obj)
                {
                    PlayLionRoarSound(false);
                }
                tmr.Enabled = false;
                tmr.Elapsed -= Tmr_Elapsed;
            }

        }


        public static void PlayLionRoarSound(bool play)
        {
            if (soundPlayer == null)
            {

                soundPlayer = new SoundPlayer($"{System.Reflection.Assembly.GetEntryAssembly().Location.Replace("PtrotectFarmByWildAnimal.dll", "lionroar.wav")}");
            }
            if (play)
            {
                soundPlayer.PlayLooping();
                Console.WriteLine($"{DateTime.Now.ToString("dd-MM-yyy-HH-mm-ss")} Playing lion roar sound");

            }
            else if (play == false)
            {
                soundPlayer.Stop();
                Console.WriteLine($"{DateTime.Now.ToString("dd-MM-yyy-HH-mm-ss")} Stopped lion roar sound");

            }
        }


        public static async Task GetCurrentSoundPlayingStatusAsync(string connectionstring)
        {
            try
            {
                CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionstring);

                CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

                CloudTable _linkTable = tableClient.GetTableReference(TableName);

                ManorMonkeyDeatails manorMonkeyDeatails = new ManorMonkeyDeatails();

                TableQuery<ManorMonkeyDeatails> query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, status));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await _linkTable.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    manorMonkeyDeatails = resultSegment.Results.FirstOrDefault();


                    CurrentSoundPlayingStatus = manorMonkeyDeatails.SoundPlayingStatus;


                } while (token != null);


            }
            catch (Exception exp)
            {
                Debug.Write(exp);

            }
        }


        public static async Task<bool> UploadFileToAzureStorageAsync(Stream filestream, string storageaccounturi, string connectionstring, string containername, string filename)
        {
            Uri bloburi = new Uri($"{storageaccounturi}/{containername}/{filename}");
            BlobClient blobClient = new BlobClient(connectionstring, containername, filename);
            await blobClient.UploadAsync(filestream);
            return await Task.FromResult(true);
        }

    }
}

Building Azure Function Server less app

Azure Function :- Azure Functions is a server less solution that allows you to write less code, maintain less infrastructure, and save on costs. Instead of worrying about deploying and maintaining servers, the cloud infrastructure provides all the up-to-date resources needed to keep your applications running.

You focus on the pieces of code that matter most to you, and Azure Functions handles the rest.

Learn more here

Blob Trigger function will execute the code once blob is uploaded to specified storage account's container while creating function we will supply correct values to it as below steps.

now go to solution explorer in visual studio click on solution and add new project

it will ask for the name of project enter custom vision then click on create

then click on browse and select azure storage account you created

then enter the name of connection string "cn" and path of container "sample-item" we have created replace with actual values you have

click on create

and replace the code with below code generated by Azure function
also install below Nuget packages from tools -> Nuget package manager -> Manage Nuget for solution

Microsoft.Azure.WebJobs;
Microsoft.Extensions.Logging;
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction;
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training;
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models;
Microsoft.WindowsAzure.Storage;
Microsoft.WindowsAzure.Storage.Table;

ManOrMonkeyFunc.cs


using System;
using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using System.Net.Http;
using System.Diagnostics;

namespace IdentifyManOrMonkeyCustomVision
{
    public static class ManOrMonkeyFunc
    {

        private static string trainingEndpoint = "enter api end point";
        private static string trainingKey = "enter key here";

        private static string predictionEndpoint = "enter api end point";
        private static string predictionKey = "enter key here";


        private static string publishedModelName = "Iteration1";

        private static string StorageAccountURIWithConatinerName = "enter storage account uri with containername";

        private static string ProjectGUID = "enter project guid";

        private static string ManTagname = "man";

        private static string MonkeyTagname = "monkey";

        private static string TableName = "helpfarmer";

        private const string playsound = "playing";

        private const string status = "status";

        private static TimeZoneInfo INDIAN_ZONE = TimeZoneInfo.FindSystemTimeZoneById("India Standard Time");

        static ManorMonkeyDeatails CurrentManorMonkeyDeatails;


        static CloudStorageAccount storageAccount = CloudStorageAccount.Parse("enter connection string here");

        static CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        static CloudTable table = tableClient.GetTableReference(TableName);


        [FunctionName("ManOrMonkeyFunc")]
        public async static Task Run([BlobTrigger("manormonkey/{name}", Connection = "AzureWebJobsStorage")] Stream myBlob, string name, ILogger log)
        {

            CustomVisionTrainingClient trainingApi = AuthTraining(trainingEndpoint, trainingKey);
            CustomVisionPredictionClient predictionApi = AuthPrediction(predictionEndpoint, predictionKey);

            Project project = GetExistingProject(trainingApi);

            var response = await TestManORMonkeyPrediction(predictionApi, project, $"{StorageAccountURIWithConatinerName}{name}");

            if (response.monkey > response.man)
            {
                await InsertIncidentgDeatilsTOAzureTable($"Please review recent image looks like monkeys are entering into the farm probability is {response.monkey:P1}", $"{StorageAccountURIWithConatinerName}{name}", log);

                await InsertIncidentgDeatilsTOAzureTable(string.Empty, string.Empty, log, true);
            }

        }




        public static async Task<bool> InsertIncidentgDeatilsTOAzureTable(string message, string imageurl, ILogger log, bool insertstatus = false)
        {
            try
            {


                ManorMonkeyDeatails details= null;

                if (insertstatus)
                {
                    await GetCurrentSoundPlayingStatusAsync();

                    CurrentManorMonkeyDeatails.SoundPlayingStatus = playsound;

                    CurrentManorMonkeyDeatails.IncidentTime = DateTime.Now;

                    TableOperation updateoperation = TableOperation.Replace(CurrentManorMonkeyDeatails);
                }
                else
                {

                    details = new ManorMonkeyDeatails($"{TableName}", $"{TableName}{DateTime.Now:dd-MM-yyyy-HH-mm-ss}");

                    details.IncidentTime = TimeZoneInfo.ConvertTimeFromUtc(DateTime.UtcNow, INDIAN_ZONE);

                    details.Message = message;
                    details.ImageURL = imageurl;
                }



                TableOperation tblops = null;

                TableResult operationresult = null;

                if (insertstatus)
                {
                    tblops = TableOperation.Replace(CurrentManorMonkeyDeatails);

                    operationresult = await table.ExecuteAsync(tblops);
                }
                else
                {
                    tblops = TableOperation.Insert(details);

                    operationresult = await table.ExecuteAsync(tblops);

                }

                var sts = operationresult.HttpStatusCode;

                return true;
            }
            catch (Exception ex)
            {
                log.LogError(ex.ToString());
                return default;
            }
        }



        public static async Task GetCurrentSoundPlayingStatusAsync()
        {
            try
            {
                TableQuery<ManorMonkeyDeatails> query;




                query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, status));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    CurrentManorMonkeyDeatails = resultSegment.Results.FirstOrDefault();


                } while (token != null);



            }
            catch (Exception exp)
            {
                Debug.Write(exp);

            }
        }

        private static CustomVisionTrainingClient AuthTraining(string endpoint, string trainingKey)
        {

            CustomVisionTrainingClient trainingApi = new CustomVisionTrainingClient(new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.ApiKeyServiceClientCredentials(trainingKey))
            {
                Endpoint = endpoint
            };
            return trainingApi;
        }
        private static CustomVisionPredictionClient AuthPrediction(string endpoint, string predictionKey)
        {

            CustomVisionPredictionClient predictionApi = new CustomVisionPredictionClient(new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.ApiKeyServiceClientCredentials(predictionKey))
            {
                Endpoint = endpoint
            };
            return predictionApi;
        }


        private static Project GetExistingProject(CustomVisionTrainingClient trainingApi)
        {
            return trainingApi.GetProject(Guid.Parse(ProjectGUID));
        }

        private async static Task<(double man, double monkey)> TestManORMonkeyPrediction(CustomVisionPredictionClient predictionApi, Project project, string bloburi)
        {

            Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.Models.ImageUrl imageUrl = new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.Models.ImageUrl(bloburi);
            var result = await predictionApi.ClassifyImageUrlAsync(project.Id, publishedModelName, imageUrl);

            double manprob = result.Predictions.Where(x => x.TagName == ManTagname).FirstOrDefault().Probability;

            double monkeyprob = result.Predictions.Where(x => x.TagName == MonkeyTagname).FirstOrDefault().Probability;

            return (manprob, monkeyprob);

        }

    }
}

ManorMonkeyDeatails.cs


using System;
using Microsoft.WindowsAzure.Storage.Table;

namespace IdentifyManOrMonkeyCustomVision
{
    public class ManorMonkeyDeatails : TableEntity
    {
        public ManorMonkeyDeatails()
        {

        }
        public ManorMonkeyDeatails(string skey, string srow)
        {
            this.PartitionKey = skey;
            this.RowKey = srow;
        }
        public DateTime IncidentTime { get; set; }

        public string Message { get; set; }

        public string ImageURL { get; set; }

        public string SoundPlayingStatus { get; set; }

    }
}

Xamarin:- Xamarin is an open-source platform for building modern and performant applications for iOS, Android, and Windows with .NET. Xamarin is an abstraction layer that manages communication of shared code with underlying platform code. Xamarin runs in a managed environment that provides conveniences such as memory allocation and garbage collection.

More details for Xamarin can be found here Learn More about Xamarin

Click on solution the click on new project as below

then enter project name as ProtectFarm

and select below shown Tabbed Option

Click on create

once it is created go to Views folder and delete existing views right click and add new content view name ProtectFarm.xaml

ProtectFarm.xaml


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="ProtectFarm.Views.ProtectFarmPage"
             xmlns:vm="clr-namespace:ProtectFarm.ViewModels"
             Title="{Binding Title}">

    <ContentPage.BindingContext>
        <vm:ProtectFarmViewModel />
    </ContentPage.BindingContext>

    <ContentPage.Resources>
        <ResourceDictionary>
            <Color x:Key="Accent">#96d1ff</Color>
        </ResourceDictionary>
    </ContentPage.Resources>
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto" />
            <RowDefinition Height="*" />
        </Grid.RowDefinitions>
        <StackLayout BackgroundColor="{StaticResource Accent}" VerticalOptions="FillAndExpand" HorizontalOptions="FillAndExpand">
            <StackLayout Orientation="Horizontal" Padding="10" HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand">

                <Label Text="Sound Status" HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand"></Label>

                <Label Text="{Binding CurrentSoundPlayingStatus}" HorizontalOptions="FillAndExpand"  VerticalOptions="FillAndExpand"  ></Label>


                <Label Text="Update Time" HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand"></Label>

                <Label Text="{Binding IncidentDateTime}" HorizontalOptions="FillAndExpand"  VerticalOptions="FillAndExpand" ></Label>

            </StackLayout>
        </StackLayout>
        <ScrollView Grid.Row="1">
            <StackLayout Orientation="Vertical" Margin="0,20,0,0" Padding="10" Spacing="10">


                <StackLayout  Orientation="Horizontal">

                    <Button Text="Play Sound" x:Name="startsound" 
                        Command="{Binding PlaySoundCommand}"
                        BackgroundColor="{StaticResource Accent}"
                        TextColor="White"  HorizontalOptions="FillAndExpand"></Button>

                    <Button Text="Stop Sound" x:Name="stopdound" 
                        Command="{Binding StopSoundCommand}"
                        BackgroundColor="{StaticResource Accent}" 
                        TextColor="White"  HorizontalOptions="FillAndExpand"></Button>

                </StackLayout>

            </StackLayout>
        </ScrollView>
    </Grid>
</ContentPage>

ItemsPage.xaml



<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="ProtectFarm.Views.ItemsPage"
             Title="{Binding Title}"
             xmlns:local="clr-namespace:ProtectFarm.ViewModels"  
             xmlns:model="clr-namespace:ProtectFarm.Models"  
             x:Name="BrowseItemsPage">

    <ContentPage.ToolbarItems>
        <!--<ToolbarItem Text="Add" Command="{Binding AddItemCommand}" />-->
    </ContentPage.ToolbarItems>
    <!--
      x:DataType enables compiled bindings for better performance and compile time validation of binding expressions.
      https://docs.microsoft.com/xamarin/xamarin-forms/app-fundamentals/data-binding/compiled-bindings
    -->
    <RefreshView x:DataType="local:ItemsViewModel" Command="{Binding LoadItemsCommand}" IsRefreshing="{Binding IsBusy, Mode=TwoWay}">
        <CollectionView x:Name="ItemsListView"
                ItemsSource="{Binding Items}"
                SelectionMode="None">
            <CollectionView.ItemTemplate>
                <DataTemplate>
                    <StackLayout Padding="10" Orientation="Horizontal" x:DataType="model:ManorMonkeyDeatails">
                        <StackLayout Orientation="Vertical">
                        <Label Text="{Binding IncidentTime}" 
                            LineBreakMode="NoWrap" 
                            Style="{DynamicResource ListItemTextStyle}" 
                            FontSize="16" />
                        <Label Text="{Binding Message}" 
                            LineBreakMode="WordWrap" WidthRequest="375"
                            Style="{DynamicResource ListItemDetailTextStyle}"
                            FontSize="13" />
                        </StackLayout>
                        <Image Source="{Binding ImageURL}"  HorizontalOptions="EndAndExpand" HeightRequest="100" WidthRequest="100"></Image>
                        <StackLayout.GestureRecognizers>
                            <TapGestureRecognizer 
                                NumberOfTapsRequired="1"
                                Command="{Binding Source={RelativeSource AncestorType={x:Type local:ItemsViewModel}}, Path=ItemTapped}"     
                                CommandParameter="{Binding .}">
                            </TapGestureRecognizer>
                        </StackLayout.GestureRecognizers>
                    </StackLayout>
                </DataTemplate>
            </CollectionView.ItemTemplate>
        </CollectionView>
    </RefreshView>
</ContentPage>

ItemDetailPage.xaml


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="ProtectFarm.Views.ItemDetailPage"
             Title="{Binding Title}">

    <StackLayout Spacing="20" Padding="15">
        <Image Source="{Binding IMGURL}" HeightRequest="250" WidthRequest="250" ></Image>
    </StackLayout>

</ContentPage>

go to ViewModels from solution explorer and add new viewmodel class as below

ProtectFarmViewModel.cs


using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using System.Windows.Input;
using Xamarin.Essentials;
using Xamarin.Forms;
using System.Linq;
using System.Linq.Expressions;
using ProtectFarm.Models;

namespace ProtectFarm.ViewModels
{
    public class ProtectFarmViewModel : BaseViewModel
    {

        private string curretstatus;

        public string CurrentSoundPlayingStatus
        {
            get => curretstatus;
            set => SetProperty(ref curretstatus, value);
        }


        private DateTime incdt;

        public DateTime IncidentDateTime
        {
            get => incdt;
            set => SetProperty(ref incdt, value);
        }

        const string status = "status";

        private const string playsound = "playing";

        private const string stopsound = "stopped";

        public ProtectFarmViewModel()
        {
            Title = "Protect Farm";
            PlaySoundCommand = new Command(async () => await UpdateSoundPlayingStatustoAzureTable(playsound).ConfigureAwait(false));
            StopSoundCommand = new Command(async () => await UpdateSoundPlayingStatustoAzureTable(stopsound).ConfigureAwait(false));

            Task.Run(async () => await GetCurrentSoundPlayingStatusAsync().ConfigureAwait(false));
        }

        public ICommand PlaySoundCommand { get; }

        public ICommand StopSoundCommand { get; }



        public async Task GetCurrentSoundPlayingStatusAsync()
        {
            try
            {
                TableQuery<ManorMonkeyDeatails> query;


                query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, status));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await _linkTable.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    manorMonkeyDeatails = resultSegment.Results.FirstOrDefault();


                    CurrentSoundPlayingStatus = manorMonkeyDeatails.SoundPlayingStatus;
                    IncidentDateTime = manorMonkeyDeatails.IncidentTime;

                } while (token != null);

            }
            catch (Exception exp)
            {
                Debug.Write(exp);

            }
        }


        public async Task UpdateSoundPlayingStatustoAzureTable(string command)
        {
            if (manorMonkeyDeatails == null)
            {
                await GetCurrentSoundPlayingStatusAsync().ConfigureAwait(false);
            }

            manorMonkeyDeatails.SoundPlayingStatus = command;

            manorMonkeyDeatails.IncidentTime = DateTime.Now;

            TableOperation updateoperation = TableOperation.Replace(manorMonkeyDeatails);

            var insertoperationresult = await _linkTable.ExecuteAsync(updateoperation);

            CurrentSoundPlayingStatus = command;

        }

    }
}

ItemsViewModel.cs


using Microsoft.WindowsAzure.Storage.Table;
using ProtectFarm.Models;
using ProtectFarm.Views;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace ProtectFarm.ViewModels
{
    public class ItemsViewModel : BaseViewModel
    {

        private string imgurl;

        public string ImgURL
        {
            get => imgurl;
            set => SetProperty(ref imgurl, value);
        }


        private string msg;

        public string MSG
        {
            get => msg;
            set => SetProperty(ref msg, value);
        }



        private DateTime     incdt;

        public DateTime Incident
        {
            get => incdt;
            set => SetProperty(ref incdt, value);
        }




        private ManorMonkeyDeatails _selectedItem;

        public ObservableCollection<ManorMonkeyDeatails> Items { get; }
        public Command LoadItemsCommand { get; }
        public Command AddItemCommand { get; }
        public Command<ManorMonkeyDeatails> ItemTapped { get; }

        public ItemsViewModel()
        {
            Title = "History";
            Items = new ObservableCollection<ManorMonkeyDeatails>();
            LoadItemsCommand = new Command(async () => await ExecuteLoadItemsCommand());

            ItemTapped = new Command<ManorMonkeyDeatails>(OnItemSelected);

            //AddItemCommand = new Command(OnAddItem);
        }

        async Task ExecuteLoadItemsCommand()
        {
            IsBusy = true;

            try
            {
                Items.Clear();
                var items = await GetHistoryAsync().ConfigureAwait(false);
                foreach (var item in items)
                {
                    Items.Add(item);
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex);
            }
            finally
            {
                IsBusy = false;
            }
        }

        public void OnAppearing()
        {
            IsBusy = true;
            SelectedItem = null;
        }

        public ManorMonkeyDeatails SelectedItem
        {
            get => _selectedItem;
            set
            {
                SetProperty(ref _selectedItem, value);
                OnItemSelected(value);
            }
        }



        async void OnItemSelected(ManorMonkeyDeatails item)
        {
            if (item == null)
                return;

            // This will push the ItemDetailPage onto the navigation stack
            await Shell.Current.GoToAsync($"{nameof(ItemDetailPage)}?{nameof(ItemDetailViewModel.IMGURL)}={item.ImageURL}");
        }



        public async Task<List<ManorMonkeyDeatails>> GetHistoryAsync()
        {
            try
            {
                List<ManorMonkeyDeatails> manorMonkeyDeatailslist = new List<ManorMonkeyDeatails>();

                TableQuery<ManorMonkeyDeatails> query;


                query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, $"{TableName}"));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await _linkTable.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    foreach (var entity in resultSegment.Results)
                    {
                        ManorMonkeyDeatails details = new ManorMonkeyDeatails
                        {
                            IncidentTime = entity.IncidentTime,
                            ImageURL = entity.ImageURL,
                            Message = entity.Message

                        };

                        manorMonkeyDeatailslist.Add(details);
                    }
                } while (token != null);


                return manorMonkeyDeatailslist;

            }
            catch (Exception exp)
            {
                Debug.Write(exp);
                return default;
            }
        }
    }
}

ItemDetailViewModel.cs


using ProtectFarm.Models;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace ProtectFarm.ViewModels
{
    [QueryProperty(nameof(IMGURL), nameof(IMGURL))]
    public class ItemDetailViewModel : BaseViewModel
    {

        private string imgurl;

        public string Id { get; set; }

        public string IMGURL
        {
            get => imgurl;
            set => SetProperty(ref imgurl, value);
        }

    }
}

BaseViewModel.cs


using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using ProtectFarm.Models;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Runtime.CompilerServices;
using Xamarin.Forms;

namespace ProtectFarm.ViewModels
{
    public class BaseViewModel : INotifyPropertyChanged
    {
        internal static string TableName = "helpfarmer";


        //please enter correct values from azure portal
        internal static CloudStorageAccount storageAccount = CloudStorageAccount.Parse("enter connection string here");

        internal static CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        internal static CloudTable _linkTable = tableClient.GetTableReference(TableName);

        internal static ManorMonkeyDeatails manorMonkeyDeatails = new ManorMonkeyDeatails();



        bool isBusy = false;
        public bool IsBusy
        {
            get { return isBusy; }
            set { SetProperty(ref isBusy, value); }
        }

        string title = string.Empty;
        public string Title
        {
            get { return title; }
            set { SetProperty(ref title, value); }
        }

        protected bool SetProperty<T>(ref T backingStore, T value,
            [CallerMemberName] string propertyName = "",
            Action onChanged = null)
        {
            if (EqualityComparer<T>.Default.Equals(backingStore, value))
                return false;

            backingStore = value;
            onChanged?.Invoke();
            OnPropertyChanged(propertyName);
            return true;
        }

        #region INotifyPropertyChanged
        public event PropertyChangedEventHandler PropertyChanged;
        protected void OnPropertyChanged([CallerMemberName] string propertyName = "")
        {
            var changed = PropertyChanged;
            if (changed == null)
                return;

            changed.Invoke(this, new PropertyChangedEventArgs(propertyName));
        }
        #endregion
    }
}

AppShell.xaml


<?xml version="1.0" encoding="UTF-8"?>
<Shell xmlns="http://xamarin.com/schemas/2014/forms" 
       xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
       xmlns:local="clr-namespace:ProtectFarm.Views"
       Title="ProtectFarm"
       x:Class="ProtectFarm.AppShell">

    <!--
        The overall app visual hierarchy is defined here, along with navigation.

        https://docs.microsoft.com/xamarin/xamarin-forms/app-fundamentals/shell/
    -->

    <Shell.Resources>
        <ResourceDictionary>
            <Style x:Key="BaseStyle" TargetType="Element">
                <Setter Property="Shell.BackgroundColor" Value="{StaticResource Primary}" />
                <Setter Property="Shell.ForegroundColor" Value="White" />
                <Setter Property="Shell.TitleColor" Value="White" />
                <Setter Property="Shell.DisabledColor" Value="#B4FFFFFF" />
                <Setter Property="Shell.UnselectedColor" Value="#95FFFFFF" />
                <Setter Property="Shell.TabBarBackgroundColor" Value="{StaticResource Primary}" />
                <Setter Property="Shell.TabBarForegroundColor" Value="White"/>
                <Setter Property="Shell.TabBarUnselectedColor" Value="#95FFFFFF"/>
                <Setter Property="Shell.TabBarTitleColor" Value="White"/>
            </Style>
            <Style TargetType="TabBar" BasedOn="{StaticResource BaseStyle}" />
            <Style TargetType="FlyoutItem" BasedOn="{StaticResource BaseStyle}" />
        </ResourceDictionary>
    </Shell.Resources>

    <TabBar>
        <ShellContent Title="Protect Farm" Icon="icon_about.png"  ContentTemplate="{DataTemplate local:ProtectFarmPage}" />
        <ShellContent Title="History" Icon="icon_feed.png" ContentTemplate="{DataTemplate local:ItemsPage}" />
    </TabBar>

    <!--
        If you would like to navigate to this content you can do so by calling
        await Shell.Current.GoToAsync("//LoginPage");
    -->



</Shell>

Also last but no least I have used Azure custom vision to build the app which can be accessed from the https://www.customvision.ai/ with active azure subscription

We have to sign in with Microsoft account with azure subscription then click on new project

we need to enter the resource details on web page and have select General (compact) which allow us to export our prediction project to Tensorflow, CoreML, ONNX model which we can use once project training has been completed and we can extend our model to offline scenario as well device running with android , ios to process our images to make prediction

on the webpage we need to add tag here i added man and monkey as 2 tags and trained project using around 20 images per tag and ran quick training then published the project

it has 3 tab we can upload image and set tag on each image from webpage it self after upload then we can click on train image.

Once Training has completed we can get Prediction url from Performance Tab which is used in Azure Function Project.

Please see video in Live action

https://www.youtube.com/watch?v=ovf45SnUp_E

Thanks a lot :) to read my story .