DEV Community

loading...

Protect Farm from Wild Animal using Azure cognitive services Custom Vision, Azure Functions, Table, .Net 5 , Xamarin App

jaymin93 profile image jaymin93 ・16 min read
  • Problem statement

Farmers are facing real challenges with the Crop attacked by Wild Animals , Most of the times Monkeys are biggest threat or Problem for them since once they attack on farm they eat and damage crop and causes them huge loss of ready crop and fruits that cost them financially loss.

sad farmer

Even in some case many farmers has left farming and switch to other business due to frequent attack of monkeys and financially loss.

monkey omg

refer link below for same kind of issues farmer facing in real life

https://english.mathrubhumi.com/agriculture/agri-news/these-farmers-of-puralimala-valley-gave-up-farming--1.4645479

https://www.downtoearth.org.in/coverage/wildlife-biodiversity/out-of-control-why-monkeys-are-a-menace-50817

https://indianexpress.com/article/india/india-news-india/man-vs-monkey-how-farmers-are-grappling-with-losses-that-have-hit-himachals-economy/

This incident of monkey attacked farm happened to my relative and costed them huge financially loss due to Monkey's 25-30 group attack on his farm , I came to know via the conversation with him , he also told that they could have took the action and make them run away from farm if somebody was aware that monkey group has attacked to Farm .

  • Solution

here is my idea

So to solve out this real world Problem idea came into my mind as I was using many azure services already.
Azure cognitive services Custom Vision can help me here to identity the entrance of monkey from the captured image of farm and I can take preventive action against the entrance of Monkey by Playing Lion roar sound also alerting farmers for the incident which is effective solution and can help to protect farm.

here is my idea

πŸ‘©β€πŸŒΎProtect Farmer 🐡 is application developed using the Azure cognitive services Custom Vision , Azure Functions with Blob Trigger, Azure Table , .Net 5 Console App, Xamarin App (UWP ,Android , Ios ) to display captured image of farm with presence of Monkey existence and take preventive action.

Here is the Link of working App , Just Enter correct storage account table , storage and custom vision resource value to run it

https://github.com/jaymin93/PtrotectFarm

  • Explanation of the concepts applied

Application Flow

Alt Text

Protect Farmer has 3 Solutions

1 .Net 5 Console App :- which will monitor for new captured images from CCTV or Surveillance camera , once it has new image captured it will upload to Azure Blob Storage

Also .net 5 Console App will help to play lion roar sound once it has retrieved the monkey detection

2 Azure Function Blob Trigger :- Azure Function with Blob Trigger , it will be triggered once new image uploaded to blob storage.

It will be sent to Azure cognitive services Custom vision
for identifying the existence of Monkey in the captured image

If Monkey existence found in captured image Azure Function will insert details to Azure Table along with Lion roar playing sound status updated to Azure Table

3 Xamarin App - UWP , Android , Ios :- Xamarin App display's Captured image that let's review farmer the Monkey existence with Result from Azure cognitive services Custom Vision stored in Azure Table also controls the Lion Roar Sound

show me a code

Let's dive in to the code and how to build each solution.

what you need is

Visual Studio 2019 :- get visual studio 2019
https://visualstudio.microsoft.com/downloads/
there is community edition which is free.

Azure Account :- you can get azure account from here

https://azure.microsoft.com/en-us/free/

And there is 25 services which are always free , if you are student Microsoft offers 100 USD credits as detailed here
Get 100 USD Student Credit

https://azure.microsoft.com/en-us/free/students/

.Net 5 SDK :- get .Net 5 SDK Here https://dotnet.microsoft.com/download/dotnet/5.0

Xamarin and Azure Function workload for visual studio :- Please install workloads shown in red box as part of your visual studio 2019 installation

Alt Text

Alt Text

Building .NET 5 Console App

Start Visual Studio 2019, Click on Create New Project as shown in red box

Alt Text

click on start new project

Alt Text

Next screen will ask you for project name and directory please select directory and provide name PtrotectFarmByWildAnimal, once it has been created go to Program.cs and replace with following code

And Intsall following Nuget Packages as below

Nuget packages from tools -> Nuget package manager -> Manage Nuget for solution

Azure.Storage.Blobs;
Microsoft.Extensions.Configuration;
Microsoft.Extensions.Hosting;
Microsoft.WindowsAzure.Storage;
Microsoft.WindowsAzure.Storage.Table;

In App settings it requires connectionstring , storageaccounturi , containername which can be retrieved from the azure blob storage account .

To create Azure Storage Account go to https://portal.azure.com/

Alt Text

Alt Text

enter resource group name or select existing and add storage account name

click on networking , data protection , advance , tags keep default selection and click on review and create button , once storage account is created it will be displayed in storage account list .

Go to storage account you have created and select access key tab as below image

Alt Text

then click on show key and from there you can get connection string ,

go to storage explorer preview and create new blob container , notice container name and storage account uri and set correct values in appsetting.json

Alt Text

Program.cs


using Azure.Storage.Blobs;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Hosting;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using ProtectFarm.Models;
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;
using System.Timers;
using System.Linq;
using System.Linq.Expressions;
using System.Diagnostics;
using System.Media;
using System.Linq.Expressions;

namespace ProtectFarmers
{
    class Program
    {
        static IConfiguration config;

        internal static string TableName = "helpfarmer";

        const string status = "status";

        private const string playsound = "playing";

        private const string stopsound = "stopped";


        public static string CurrentSoundPlayingStatus
        {
            get; set;
        }

        public static string connectionstring { get; set; }

        public static string storageaccounturi { get; set; }

        public static string containername { get; set; }

        public static Timer tmr;

        public static SoundPlayer soundPlayer;

        public static object obj = new object();

        public async static Task Main(string[] args)
        {
            HostBuilder builder = new HostBuilder();

            config = new ConfigurationBuilder()
             .AddJsonFile("appsettings.json", true, true)
             .Build();

            FileSystemWatcher fileSystemWatcher = new FileSystemWatcher();

//set correct directory path as needed.
            fileSystemWatcher.Path = "G:\\cameradir";
            fileSystemWatcher.Created += FileSystemWatcher_Created;
            fileSystemWatcher.EnableRaisingEvents = true;
            await builder.RunConsoleAsync();
        }

        private async static void FileSystemWatcher_Created(object sender, FileSystemEventArgs e)
        {
            using (FileStream fileStream = new FileStream(e.FullPath, FileMode.Open))
            {
                connectionstring = config["connectionstring"];
                storageaccounturi = config["storageaccounturi"];
                containername = config["containername"];

                await UploadFileToAzureStorageAsync(fileStream, storageaccounturi, connectionstring, containername, DateTime.Now.ToString("dd-MM-yyyy-HH-mm-ss") + e.Name);
            }

            if (tmr == null)
            {
                tmr = new Timer();
                tmr.Enabled = true;
                tmr.Interval = 15000;
                tmr.Elapsed += Tmr_Elapsed;
            }
        }

        private async static void Tmr_Elapsed(object sender, ElapsedEventArgs e)
        {
            await GetCurrentSoundPlayingStatusAsync(connectionstring);


            if (CurrentSoundPlayingStatus == playsound)
            {
                lock (obj)
                {
                    PlayLionRoarSound(true);
                }
            }
            else
            {
                lock (obj)
                {
                    PlayLionRoarSound(false);
                }
                tmr.Enabled = false;
                tmr.Elapsed -= Tmr_Elapsed;
            }

        }


        public static void PlayLionRoarSound(bool play)
        {
            if (soundPlayer == null)
            {

                soundPlayer = new SoundPlayer($"{System.Reflection.Assembly.GetEntryAssembly().Location.Replace("PtrotectFarmByWildAnimal.dll", "lionroar.wav")}");
            }
            if (play)
            {
                soundPlayer.PlayLooping();
                Console.WriteLine($"{DateTime.Now.ToString("dd-MM-yyy-HH-mm-ss")} Playing lion roar sound");

            }
            else if (play == false)
            {
                soundPlayer.Stop();
                Console.WriteLine($"{DateTime.Now.ToString("dd-MM-yyy-HH-mm-ss")} Stopped lion roar sound");

            }
        }


        public static async Task GetCurrentSoundPlayingStatusAsync(string connectionstring)
        {
            try
            {
                CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connectionstring);

                CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

                CloudTable _linkTable = tableClient.GetTableReference(TableName);

                ManorMonkeyDeatails manorMonkeyDeatails = new ManorMonkeyDeatails();

                TableQuery<ManorMonkeyDeatails> query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, status));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await _linkTable.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    manorMonkeyDeatails = resultSegment.Results.FirstOrDefault();


                    CurrentSoundPlayingStatus = manorMonkeyDeatails.SoundPlayingStatus;


                } while (token != null);


            }
            catch (Exception exp)
            {
                Debug.Write(exp);

            }
        }


        public static async Task<bool> UploadFileToAzureStorageAsync(Stream filestream, string storageaccounturi, string connectionstring, string containername, string filename)
        {
            Uri bloburi = new Uri($"{storageaccounturi}/{containername}/{filename}");
            BlobClient blobClient = new BlobClient(connectionstring, containername, filename);
            await blobClient.UploadAsync(filestream);
            return await Task.FromResult(true);
        }

    }
}



Enter fullscreen mode Exit fullscreen mode

Building Azure Function Server less app

Azure Function :- Azure Functions is a server less solution that allows you to write less code, maintain less infrastructure, and save on costs. Instead of worrying about deploying and maintaining servers, the cloud infrastructure provides all the up-to-date resources needed to keep your applications running.

You focus on the pieces of code that matter most to you, and Azure Functions handles the rest.

Learn more here

Blob Trigger function will execute the code once blob is uploaded to specified storage account's container while creating function we will supply correct values to it as below steps.

now go to solution explorer in visual studio click on solution and add new project

Alt Text

Alt Text

it will ask for the name of project enter custom vision then click on create

Alt Text

then click on browse and select azure storage account you created

Alt Text

then enter the name of connection string "cn" and path of container "sample-item" we have created replace with actual values you have

Alt Text

click on create

and replace the code with below code generated by Azure function
also install below Nuget packages from tools -> Nuget package manager -> Manage Nuget for solution

Microsoft.Azure.WebJobs;
Microsoft.Extensions.Logging;
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction;
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training;
Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models;
Microsoft.WindowsAzure.Storage;
Microsoft.WindowsAzure.Storage.Table;

ManOrMonkeyFunc.cs


using System;
using System.IO;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training;
using Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.Models;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using System.Net.Http;
using System.Diagnostics;

namespace IdentifyManOrMonkeyCustomVision
{
    public static class ManOrMonkeyFunc
    {

        private static string trainingEndpoint = "enter api end point";
        private static string trainingKey = "enter key here";

        private static string predictionEndpoint = "enter api end point";
        private static string predictionKey = "enter key here";


        private static string publishedModelName = "Iteration1";

        private static string StorageAccountURIWithConatinerName = "enter storage account uri with containername";

        private static string ProjectGUID = "enter project guid";

        private static string ManTagname = "man";

        private static string MonkeyTagname = "monkey";

        private static string TableName = "helpfarmer";

        private const string playsound = "playing";

        private const string status = "status";

        private static TimeZoneInfo INDIAN_ZONE = TimeZoneInfo.FindSystemTimeZoneById("India Standard Time");

        static ManorMonkeyDeatails CurrentManorMonkeyDeatails;


        static CloudStorageAccount storageAccount = CloudStorageAccount.Parse("enter connection string here");

        static CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        static CloudTable table = tableClient.GetTableReference(TableName);


        [FunctionName("ManOrMonkeyFunc")]
        public async static Task Run([BlobTrigger("manormonkey/{name}", Connection = "AzureWebJobsStorage")] Stream myBlob, string name, ILogger log)
        {

            CustomVisionTrainingClient trainingApi = AuthTraining(trainingEndpoint, trainingKey);
            CustomVisionPredictionClient predictionApi = AuthPrediction(predictionEndpoint, predictionKey);

            Project project = GetExistingProject(trainingApi);

            var response = await TestManORMonkeyPrediction(predictionApi, project, $"{StorageAccountURIWithConatinerName}{name}");

            if (response.monkey > response.man)
            {
                await InsertIncidentgDeatilsTOAzureTable($"Please review recent image looks like monkeys are entering into the farm probability is {response.monkey:P1}", $"{StorageAccountURIWithConatinerName}{name}", log);

                await InsertIncidentgDeatilsTOAzureTable(string.Empty, string.Empty, log, true);
            }

        }




        public static async Task<bool> InsertIncidentgDeatilsTOAzureTable(string message, string imageurl, ILogger log, bool insertstatus = false)
        {
            try
            {


                ManorMonkeyDeatails details= null;

                if (insertstatus)
                {
                    await GetCurrentSoundPlayingStatusAsync();

                    CurrentManorMonkeyDeatails.SoundPlayingStatus = playsound;

                    CurrentManorMonkeyDeatails.IncidentTime = DateTime.Now;

                    TableOperation updateoperation = TableOperation.Replace(CurrentManorMonkeyDeatails);
                }
                else
                {

                    details = new ManorMonkeyDeatails($"{TableName}", $"{TableName}{DateTime.Now:dd-MM-yyyy-HH-mm-ss}");

                    details.IncidentTime = TimeZoneInfo.ConvertTimeFromUtc(DateTime.UtcNow, INDIAN_ZONE);

                    details.Message = message;
                    details.ImageURL = imageurl;
                }



                TableOperation tblops = null;

                TableResult operationresult = null;

                if (insertstatus)
                {
                    tblops = TableOperation.Replace(CurrentManorMonkeyDeatails);

                    operationresult = await table.ExecuteAsync(tblops);
                }
                else
                {
                    tblops = TableOperation.Insert(details);

                    operationresult = await table.ExecuteAsync(tblops);

                }

                var sts = operationresult.HttpStatusCode;

                return true;
            }
            catch (Exception ex)
            {
                log.LogError(ex.ToString());
                return default;
            }
        }



        public static async Task GetCurrentSoundPlayingStatusAsync()
        {
            try
            {
                TableQuery<ManorMonkeyDeatails> query;




                query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, status));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    CurrentManorMonkeyDeatails = resultSegment.Results.FirstOrDefault();


                } while (token != null);



            }
            catch (Exception exp)
            {
                Debug.Write(exp);

            }
        }

        private static CustomVisionTrainingClient AuthTraining(string endpoint, string trainingKey)
        {

            CustomVisionTrainingClient trainingApi = new CustomVisionTrainingClient(new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training.ApiKeyServiceClientCredentials(trainingKey))
            {
                Endpoint = endpoint
            };
            return trainingApi;
        }
        private static CustomVisionPredictionClient AuthPrediction(string endpoint, string predictionKey)
        {

            CustomVisionPredictionClient predictionApi = new CustomVisionPredictionClient(new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.ApiKeyServiceClientCredentials(predictionKey))
            {
                Endpoint = endpoint
            };
            return predictionApi;
        }


        private static Project GetExistingProject(CustomVisionTrainingClient trainingApi)
        {
            return trainingApi.GetProject(Guid.Parse(ProjectGUID));
        }

        private async static Task<(double man, double monkey)> TestManORMonkeyPrediction(CustomVisionPredictionClient predictionApi, Project project, string bloburi)
        {

            Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.Models.ImageUrl imageUrl = new Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction.Models.ImageUrl(bloburi);
            var result = await predictionApi.ClassifyImageUrlAsync(project.Id, publishedModelName, imageUrl);

            double manprob = result.Predictions.Where(x => x.TagName == ManTagname).FirstOrDefault().Probability;

            double monkeyprob = result.Predictions.Where(x => x.TagName == MonkeyTagname).FirstOrDefault().Probability;

            return (manprob, monkeyprob);

        }

    }
}



Enter fullscreen mode Exit fullscreen mode

ManorMonkeyDeatails.cs


using System;
using Microsoft.WindowsAzure.Storage.Table;

namespace IdentifyManOrMonkeyCustomVision
{
    public class ManorMonkeyDeatails : TableEntity
    {
        public ManorMonkeyDeatails()
        {

        }
        public ManorMonkeyDeatails(string skey, string srow)
        {
            this.PartitionKey = skey;
            this.RowKey = srow;
        }
        public DateTime IncidentTime { get; set; }

        public string Message { get; set; }

        public string ImageURL { get; set; }

        public string SoundPlayingStatus { get; set; }

    }
}



Enter fullscreen mode Exit fullscreen mode

Xamarin:- Xamarin is an open-source platform for building modern and performant applications for iOS, Android, and Windows with .NET. Xamarin is an abstraction layer that manages communication of shared code with underlying platform code. Xamarin runs in a managed environment that provides conveniences such as memory allocation and garbage collection.

More details for Xamarin can be found here Learn More about Xamarin

Click on solution the click on new project as below

Alt Text

Alt Text

then enter project name as ProtectFarm

and select below shown Tabbed Option

Alt Text

Click on create

once it is created go to Views folder and delete existing views right click and add new content view name ProtectFarm.xaml

ProtectFarm.xaml


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="ProtectFarm.Views.ProtectFarmPage"
             xmlns:vm="clr-namespace:ProtectFarm.ViewModels"
             Title="{Binding Title}">

    <ContentPage.BindingContext>
        <vm:ProtectFarmViewModel />
    </ContentPage.BindingContext>

    <ContentPage.Resources>
        <ResourceDictionary>
            <Color x:Key="Accent">#96d1ff</Color>
        </ResourceDictionary>
    </ContentPage.Resources>
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto" />
            <RowDefinition Height="*" />
        </Grid.RowDefinitions>
        <StackLayout BackgroundColor="{StaticResource Accent}" VerticalOptions="FillAndExpand" HorizontalOptions="FillAndExpand">
            <StackLayout Orientation="Horizontal" Padding="10" HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand">

                <Label Text="Sound Status" HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand"></Label>

                <Label Text="{Binding CurrentSoundPlayingStatus}" HorizontalOptions="FillAndExpand"  VerticalOptions="FillAndExpand"  ></Label>


                <Label Text="Update Time" HorizontalOptions="FillAndExpand" VerticalOptions="FillAndExpand"></Label>

                <Label Text="{Binding IncidentDateTime}" HorizontalOptions="FillAndExpand"  VerticalOptions="FillAndExpand" ></Label>

            </StackLayout>
        </StackLayout>
        <ScrollView Grid.Row="1">
            <StackLayout Orientation="Vertical" Margin="0,20,0,0" Padding="10" Spacing="10">


                <StackLayout  Orientation="Horizontal">

                    <Button Text="Play Sound" x:Name="startsound" 
                        Command="{Binding PlaySoundCommand}"
                        BackgroundColor="{StaticResource Accent}"
                        TextColor="White"  HorizontalOptions="FillAndExpand"></Button>

                    <Button Text="Stop Sound" x:Name="stopdound" 
                        Command="{Binding StopSoundCommand}"
                        BackgroundColor="{StaticResource Accent}" 
                        TextColor="White"  HorizontalOptions="FillAndExpand"></Button>

                </StackLayout>

            </StackLayout>
        </ScrollView>
    </Grid>
</ContentPage>





Enter fullscreen mode Exit fullscreen mode

ItemsPage.xaml



<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="ProtectFarm.Views.ItemsPage"
             Title="{Binding Title}"
             xmlns:local="clr-namespace:ProtectFarm.ViewModels"  
             xmlns:model="clr-namespace:ProtectFarm.Models"  
             x:Name="BrowseItemsPage">

    <ContentPage.ToolbarItems>
        <!--<ToolbarItem Text="Add" Command="{Binding AddItemCommand}" />-->
    </ContentPage.ToolbarItems>
    <!--
      x:DataType enables compiled bindings for better performance and compile time validation of binding expressions.
      https://docs.microsoft.com/xamarin/xamarin-forms/app-fundamentals/data-binding/compiled-bindings
    -->
    <RefreshView x:DataType="local:ItemsViewModel" Command="{Binding LoadItemsCommand}" IsRefreshing="{Binding IsBusy, Mode=TwoWay}">
        <CollectionView x:Name="ItemsListView"
                ItemsSource="{Binding Items}"
                SelectionMode="None">
            <CollectionView.ItemTemplate>
                <DataTemplate>
                    <StackLayout Padding="10" Orientation="Horizontal" x:DataType="model:ManorMonkeyDeatails">
                        <StackLayout Orientation="Vertical">
                        <Label Text="{Binding IncidentTime}" 
                            LineBreakMode="NoWrap" 
                            Style="{DynamicResource ListItemTextStyle}" 
                            FontSize="16" />
                        <Label Text="{Binding Message}" 
                            LineBreakMode="WordWrap" WidthRequest="375"
                            Style="{DynamicResource ListItemDetailTextStyle}"
                            FontSize="13" />
                        </StackLayout>
                        <Image Source="{Binding ImageURL}"  HorizontalOptions="EndAndExpand" HeightRequest="100" WidthRequest="100"></Image>
                        <StackLayout.GestureRecognizers>
                            <TapGestureRecognizer 
                                NumberOfTapsRequired="1"
                                Command="{Binding Source={RelativeSource AncestorType={x:Type local:ItemsViewModel}}, Path=ItemTapped}"     
                                CommandParameter="{Binding .}">
                            </TapGestureRecognizer>
                        </StackLayout.GestureRecognizers>
                    </StackLayout>
                </DataTemplate>
            </CollectionView.ItemTemplate>
        </CollectionView>
    </RefreshView>
</ContentPage>


Enter fullscreen mode Exit fullscreen mode

ItemDetailPage.xaml


<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             x:Class="ProtectFarm.Views.ItemDetailPage"
             Title="{Binding Title}">

    <StackLayout Spacing="20" Padding="15">
        <Image Source="{Binding IMGURL}" HeightRequest="250" WidthRequest="250" ></Image>
    </StackLayout>

</ContentPage>

Enter fullscreen mode Exit fullscreen mode

go to ViewModels from solution explorer and add new viewmodel class as below

ProtectFarmViewModel.cs


using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using System.Windows.Input;
using Xamarin.Essentials;
using Xamarin.Forms;
using System.Linq;
using System.Linq.Expressions;
using ProtectFarm.Models;

namespace ProtectFarm.ViewModels
{
    public class ProtectFarmViewModel : BaseViewModel
    {

        private string curretstatus;

        public string CurrentSoundPlayingStatus
        {
            get => curretstatus;
            set => SetProperty(ref curretstatus, value);
        }


        private DateTime incdt;

        public DateTime IncidentDateTime
        {
            get => incdt;
            set => SetProperty(ref incdt, value);
        }

        const string status = "status";

        private const string playsound = "playing";

        private const string stopsound = "stopped";

        public ProtectFarmViewModel()
        {
            Title = "Protect Farm";
            PlaySoundCommand = new Command(async () => await UpdateSoundPlayingStatustoAzureTable(playsound).ConfigureAwait(false));
            StopSoundCommand = new Command(async () => await UpdateSoundPlayingStatustoAzureTable(stopsound).ConfigureAwait(false));

            Task.Run(async () => await GetCurrentSoundPlayingStatusAsync().ConfigureAwait(false));
        }

        public ICommand PlaySoundCommand { get; }

        public ICommand StopSoundCommand { get; }



        public async Task GetCurrentSoundPlayingStatusAsync()
        {
            try
            {
                TableQuery<ManorMonkeyDeatails> query;


                query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, status));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await _linkTable.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    manorMonkeyDeatails = resultSegment.Results.FirstOrDefault();


                    CurrentSoundPlayingStatus = manorMonkeyDeatails.SoundPlayingStatus;
                    IncidentDateTime = manorMonkeyDeatails.IncidentTime;

                } while (token != null);

            }
            catch (Exception exp)
            {
                Debug.Write(exp);

            }
        }


        public async Task UpdateSoundPlayingStatustoAzureTable(string command)
        {
            if (manorMonkeyDeatails == null)
            {
                await GetCurrentSoundPlayingStatusAsync().ConfigureAwait(false);
            }

            manorMonkeyDeatails.SoundPlayingStatus = command;

            manorMonkeyDeatails.IncidentTime = DateTime.Now;

            TableOperation updateoperation = TableOperation.Replace(manorMonkeyDeatails);

            var insertoperationresult = await _linkTable.ExecuteAsync(updateoperation);

            CurrentSoundPlayingStatus = command;

        }

    }
}

Enter fullscreen mode Exit fullscreen mode

ItemsViewModel.cs


using Microsoft.WindowsAzure.Storage.Table;
using ProtectFarm.Models;
using ProtectFarm.Views;
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Diagnostics;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace ProtectFarm.ViewModels
{
    public class ItemsViewModel : BaseViewModel
    {

        private string imgurl;

        public string ImgURL
        {
            get => imgurl;
            set => SetProperty(ref imgurl, value);
        }


        private string msg;

        public string MSG
        {
            get => msg;
            set => SetProperty(ref msg, value);
        }



        private DateTime     incdt;

        public DateTime Incident
        {
            get => incdt;
            set => SetProperty(ref incdt, value);
        }




        private ManorMonkeyDeatails _selectedItem;

        public ObservableCollection<ManorMonkeyDeatails> Items { get; }
        public Command LoadItemsCommand { get; }
        public Command AddItemCommand { get; }
        public Command<ManorMonkeyDeatails> ItemTapped { get; }

        public ItemsViewModel()
        {
            Title = "History";
            Items = new ObservableCollection<ManorMonkeyDeatails>();
            LoadItemsCommand = new Command(async () => await ExecuteLoadItemsCommand());

            ItemTapped = new Command<ManorMonkeyDeatails>(OnItemSelected);

            //AddItemCommand = new Command(OnAddItem);
        }

        async Task ExecuteLoadItemsCommand()
        {
            IsBusy = true;

            try
            {
                Items.Clear();
                var items = await GetHistoryAsync().ConfigureAwait(false);
                foreach (var item in items)
                {
                    Items.Add(item);
                }
            }
            catch (Exception ex)
            {
                Debug.WriteLine(ex);
            }
            finally
            {
                IsBusy = false;
            }
        }

        public void OnAppearing()
        {
            IsBusy = true;
            SelectedItem = null;
        }

        public ManorMonkeyDeatails SelectedItem
        {
            get => _selectedItem;
            set
            {
                SetProperty(ref _selectedItem, value);
                OnItemSelected(value);
            }
        }



        async void OnItemSelected(ManorMonkeyDeatails item)
        {
            if (item == null)
                return;

            // This will push the ItemDetailPage onto the navigation stack
            await Shell.Current.GoToAsync($"{nameof(ItemDetailPage)}?{nameof(ItemDetailViewModel.IMGURL)}={item.ImageURL}");
        }



        public async Task<List<ManorMonkeyDeatails>> GetHistoryAsync()
        {
            try
            {
                List<ManorMonkeyDeatails> manorMonkeyDeatailslist = new List<ManorMonkeyDeatails>();

                TableQuery<ManorMonkeyDeatails> query;


                query = new TableQuery<ManorMonkeyDeatails>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, $"{TableName}"));


                TableContinuationToken token = null;
                do
                {
                    TableQuerySegment<ManorMonkeyDeatails> resultSegment = await _linkTable.ExecuteQuerySegmentedAsync(query, token).ConfigureAwait(false);
                    token = resultSegment.ContinuationToken;

                    foreach (var entity in resultSegment.Results)
                    {
                        ManorMonkeyDeatails details = new ManorMonkeyDeatails
                        {
                            IncidentTime = entity.IncidentTime,
                            ImageURL = entity.ImageURL,
                            Message = entity.Message

                        };

                        manorMonkeyDeatailslist.Add(details);
                    }
                } while (token != null);


                return manorMonkeyDeatailslist;

            }
            catch (Exception exp)
            {
                Debug.Write(exp);
                return default;
            }
        }
    }
}

Enter fullscreen mode Exit fullscreen mode

ItemDetailViewModel.cs


using ProtectFarm.Models;
using System;
using System.Diagnostics;
using System.Threading.Tasks;
using Xamarin.Forms;

namespace ProtectFarm.ViewModels
{
    [QueryProperty(nameof(IMGURL), nameof(IMGURL))]
    public class ItemDetailViewModel : BaseViewModel
    {

        private string imgurl;

        public string Id { get; set; }

        public string IMGURL
        {
            get => imgurl;
            set => SetProperty(ref imgurl, value);
        }

    }
}



Enter fullscreen mode Exit fullscreen mode

BaseViewModel.cs


using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Table;
using ProtectFarm.Models;
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Runtime.CompilerServices;
using Xamarin.Forms;

namespace ProtectFarm.ViewModels
{
    public class BaseViewModel : INotifyPropertyChanged
    {
        internal static string TableName = "helpfarmer";


        //please enter correct values from azure portal
        internal static CloudStorageAccount storageAccount = CloudStorageAccount.Parse("enter connection string here");

        internal static CloudTableClient tableClient = storageAccount.CreateCloudTableClient();

        internal static CloudTable _linkTable = tableClient.GetTableReference(TableName);

        internal static ManorMonkeyDeatails manorMonkeyDeatails = new ManorMonkeyDeatails();



        bool isBusy = false;
        public bool IsBusy
        {
            get { return isBusy; }
            set { SetProperty(ref isBusy, value); }
        }

        string title = string.Empty;
        public string Title
        {
            get { return title; }
            set { SetProperty(ref title, value); }
        }

        protected bool SetProperty<T>(ref T backingStore, T value,
            [CallerMemberName] string propertyName = "",
            Action onChanged = null)
        {
            if (EqualityComparer<T>.Default.Equals(backingStore, value))
                return false;

            backingStore = value;
            onChanged?.Invoke();
            OnPropertyChanged(propertyName);
            return true;
        }

        #region INotifyPropertyChanged
        public event PropertyChangedEventHandler PropertyChanged;
        protected void OnPropertyChanged([CallerMemberName] string propertyName = "")
        {
            var changed = PropertyChanged;
            if (changed == null)
                return;

            changed.Invoke(this, new PropertyChangedEventArgs(propertyName));
        }
        #endregion
    }
}


Enter fullscreen mode Exit fullscreen mode

AppShell.xaml


<?xml version="1.0" encoding="UTF-8"?>
<Shell xmlns="http://xamarin.com/schemas/2014/forms" 
       xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
       xmlns:local="clr-namespace:ProtectFarm.Views"
       Title="ProtectFarm"
       x:Class="ProtectFarm.AppShell">

    <!--
        The overall app visual hierarchy is defined here, along with navigation.

        https://docs.microsoft.com/xamarin/xamarin-forms/app-fundamentals/shell/
    -->

    <Shell.Resources>
        <ResourceDictionary>
            <Style x:Key="BaseStyle" TargetType="Element">
                <Setter Property="Shell.BackgroundColor" Value="{StaticResource Primary}" />
                <Setter Property="Shell.ForegroundColor" Value="White" />
                <Setter Property="Shell.TitleColor" Value="White" />
                <Setter Property="Shell.DisabledColor" Value="#B4FFFFFF" />
                <Setter Property="Shell.UnselectedColor" Value="#95FFFFFF" />
                <Setter Property="Shell.TabBarBackgroundColor" Value="{StaticResource Primary}" />
                <Setter Property="Shell.TabBarForegroundColor" Value="White"/>
                <Setter Property="Shell.TabBarUnselectedColor" Value="#95FFFFFF"/>
                <Setter Property="Shell.TabBarTitleColor" Value="White"/>
            </Style>
            <Style TargetType="TabBar" BasedOn="{StaticResource BaseStyle}" />
            <Style TargetType="FlyoutItem" BasedOn="{StaticResource BaseStyle}" />
        </ResourceDictionary>
    </Shell.Resources>

    <TabBar>
        <ShellContent Title="Protect Farm" Icon="icon_about.png"  ContentTemplate="{DataTemplate local:ProtectFarmPage}" />
        <ShellContent Title="History" Icon="icon_feed.png" ContentTemplate="{DataTemplate local:ItemsPage}" />
    </TabBar>

    <!--
        If you would like to navigate to this content you can do so by calling
        await Shell.Current.GoToAsync("//LoginPage");
    -->



</Shell>


Enter fullscreen mode Exit fullscreen mode

Also last but no least I have used Azure custom vision to build the app which can be accessed from the https://www.customvision.ai/ with active azure subscription

We have to sign in with Microsoft account with azure subscription then click on new project

Alt Text

we need to enter the resource details on web page and have select General (compact) which allow us to export our prediction project to Tensorflow, CoreML, ONNX model which we can use once project training has been completed and we can extend our model to offline scenario as well device running with android , ios to process our images to make prediction

on the webpage we need to add tag here i added man and monkey as 2 tags and trained project using around 20 images per tag and ran quick training then published the project

Alt Text

it has 3 tab we can upload image and set tag on each image from webpage it self after upload then we can click on train image.

Once Training has completed we can get Prediction url from Performance Tab which is used in Azure Function Project.

Alt Text

Please see video in Live action

https://www.youtube.com/watch?v=ovf45SnUp_E

Thanks a lot :) to read my story .

Discussion (0)

Forem Open with the Forem app