DEV Community: Darryl Bayliss

Running a RAG powered language model on Android using MediaPipe

Darryl Bayliss — Thu, 18 Sep 2025 05:00:00 +0000

A couple of months ago I gave a talk about running on device SLMs in apps. It was well recieved and refreshing to be able to give a talk where mobile apps can gain an advantage from the rise in language model usage.

After the talk, I had a few pieces of feedback from attendees along the lines of “It would be good to see an example of an app using an SLM powered by RAG”.

These are fair comments. The tricky thing here is the lead time it takes to setup a model and show something meaningful to developers in a limited time. Fortunately, it does make for a great blog post!

So here it is! If you’re looking for steps to add a RAG powered language model to your Android app, this is the post for you!

What is RAG?

If you’re unfamilar with the term, RAG stands for Retrieval Augmented Generation. It’s a technique for language models to access external information that isn’t available in their dataset after training. This allows models to be aware of upto date information they can use to provide more accurate answers to prompts.

Let’s look at a quick example. Imagine you have two language models, one model is using RAG to retrieve external information about capital cities in Europe whilst the other is only relying on its own knowledge. You decide to give both models the following prompt:

You are an expert on the geography of Europe. 

Give me the name of 3 capital cities.

Also give me an interesting fact for each of them.

The result could look something like this:

As the diagram illustrates, you’re more likely to get a useful answer from the language model using RAG, instead of the model relying on its own knowledge. You may also notice less hallucinations from the RAG powered model, as it doesn’t need to make up information.

That’s really all you need to know about RAG at a high level. Next, let’s go deeper and write some code to enable your own RAG powered language model inside your app!

Setting up your Android App

Let’s say you want to create an app using a language model to play the game of Simon Says. You want the model to be Simon, and use RAG to access a datasource of tasks to help decide what to ask the user. How would you do that?

The most straight forward way to do that on Android is with MediaPipe, a collection of tools to help your app use AI and machine learning techniques. To begin, add the MediaPipe dependency to your app build.gradle:

dependencies {
  implementation("com.google.mediapipe:tasks-genai:0.10.27")
  implementation("com.google.ai.edge.localagents:localagents-rag:0.3.0")
}

Next, you need to add a language model to your test device via your computer. For this example we’ll use Google’s Gemma3-1B, a lightweight language model that holds 1 Billion parameters worth of information.

Side Note: Before downloading the model, you may have to sign upto Kaggle and agree to Google’s AI Terms and Conditions.

Once the model is downloaded. It’s time to add the model to your device. You can do this via adb:

$ adb shell mkdir -p /data/local/tmp/slm/ # Create a folder to store the model
$ adb push output_path /data/local/tmp/slm/gemma3-1B-it-int4.task # Copy the model over to the file

Alternatively, you can use the File Explorer using Android Studio to create the folders yourself and drag the model onto your device.

With the model added you can continue building the RAG pipeline to feed Gemma with information. Next, let’s look at adding the information gemma will rely to answer prompts.

Creating Embeddings

The information language models rely on to perform RAG is not the same information you pass into it. Models require information to be in a specific format called Embeddings.

These are mathematical strings that represent the text’s semantic meaning. When the model recieves a prompt it will use it to search for the most relevant information matching it, and use it alongside its own information to provide an answer. These mathematical strings are created by a tool called an Embedder.

Embeddings are a whole subject on their own you are encouraged to read about. For this post you only need to know how to create them. You can do this on device by using the Gecko Embedder.

First, download the sentencepiece.model tokenizer and the Gecko_256_f32.tflite embedder model files to your computer. Then push them to your device:

$ adb push sentencepiece.model /data/local/tmp/slm/sentencepiece.model # Push the tokenizer to the device
$ adb push Gecko_256_f32.tflite /data/local/tmp/slm/Gecko_256_f32.tflite # Push the embedder model to the device

With the embedder installed to your device, it’s time to provide a sample file to create the embeddings from. In Android Studio, in your app module assets folder, create a file called simon_says_responses.txt. Then in the file add the following text:

<chunk_splitter>
Go for a walk
<chunk_splitter>
Jump and down 10 times
<chunk_splitter>
Sing your favourite song!
<chunk_splitter>
Text your best friend a funny meme
<chunk_splitter>
Do 10 press ups!
<chunk_splitter>

The file contains a couple of different responses one could give in a game of Simon Says, each split up with a <chunk_splitter> tag. This gives the embedder a signal to know how to separate each response when splitting the text into embeddings.

This process is called chunking , and can have a large effect on how well RAG performs via the language model. Experimentation with different sized chunks and responses is encouraged to find the right combination for your needs!

One thing to consider is app storage. Remember you’ve already installed a language model and an embedder onto your device. These take up gigabytyes of space, so make sure not to further bloat a device by using a text file that is too large!

You want to consider storing the sample file in the cloud and downloading it via the network to reduce issues with storage.

With the text file in place, it’s time to initialise the embedder:


private const val GeckoEmbedderPath = "/data/local/tmp/slm/gecko_256_f32.tflite"
private const val TokenizerModelPath = "/data/local/tmp/slm/sentencepiece.model"
private const val UseGpuForEmbeddings = true

val embedder: Embedder<String> = GeckoEmbeddingModel(
    GeckoEmbedderPath,
    TokenizerModelPath,
    UseGpuForEmbeddings,
  )

The embedder takes three parameters, the path of the embedder and the tokenizer on the device, with a final parameter to set whether the embedder can use the device GPU when creating embeddings.

Setting this to true will speed up the creation of the embeddings if a GPU is available. Make sure to check the device capabilities before deciding to enable this value.

Next, create an instance of the language model:

private const val GemmaModelPath = "/data/local/tmp/slm/gemma3-1B-it-int4.task"

val llmInferenceOptions = LlmInferenceOptions.builder()
            .setModelPath(GemmaModelPath)
            .setPreferredBackend(LlmInference.Backend.CPU) // Change to GPU if you have a GPU powered device.
            .setMaxTokens(1200)
            .build()

val llmInferenceSessionOptions = LlmInferenceSessionOptions.builder()
            .setTemperature(0.6f)
            .setTopK(5000)
            .setTopP(1f)
            .build()

val languageModel = MediaPipeLlmBackend(
            context, // This is the application context
            languageModelOptions,
            languageModelSessionOptions)

    languageModel.initialize().get()

There are alot of parameters involved above, don’t worry about these for now. We’ll come back to them later on.

With the language model created, you can focus back on the embeddings. The model needs a place to retrieve the embeddings from each time it recieves a prompt.

MediaPipe provides an SQLite Vector Store , which is a common tool to store embeddings. Let’s create one:

private const val PromptTemplate: String = """
    You are Simon in a game of Simon Says.

    Your task is to ask the player to perform a task from the following list: {0}.

    Your response must only contain the task that the player must do.

    Your response must be based on the players request: {1}. 

    Do not ask the player to do the same thing twice.

    You must not ask the player to do anything that is dangerous, unethical or unlawful.
"""

val chainConfig = ChainConfig.create(
    languageModel, PromptBuilder(PromptTemplate),
    DefaultSemanticTextMemory(
    SqliteVectorStore(768), embedder
    )
)

Here, everything begins to link together as the language model and embedder are both passed into the ChainConfig. The 786 is the size of each “vector” the database can store. The PromptBuilder is used to provide a prompt to help drive the RAG process.

Finally, let’s load the text file we created earlier in the assets folder into the embedder. First, load the file from the device and split the text into a list of strings:

// This is an extension function to read the file from disk.
val gameResponses: List<String> = context.getTextFromFile("simon_says_responses.txt")

Next, load the responses into the chainConfig created earlier.

chainConfig.semanticMemory.getOrNull()
    ?.recordBatchedMemoryItems(ImmutableList.copyOf(gameResponses))
    ?.get()

Depending on your device and the size of the text file this can take a couple of seconds to a few minutes complete. A good rule of thumb is to prevent the language model from being used whilst this occurs. You could also run this operation on an IO thread to avoid blocking the main thread.

With that done, you’ve just converted your textfile into a set of embeddings kept in a vector store. You’ve also linked your language model to the store so it can now retrieve information!

The next section will show how to use those embeddings by passing the language model your prompts.

Passing Prompts to your RAG Powered Model

With most of the complex setup complete, passing a prompt into the language model is surprisingly easy. First you need to create a RetrievalAndInferenceChain using the chain config and invoke it.

val retrievalAndInferenceChain = RetrievalAndInferenceChain(chainConfig)

Next, create a request and pass it into the chain.

val prompt = "Tell me something to do Simon involving jumping!"

val retrievalRequest =
    RetrievalRequest.create(
        prompt,
        RetrievalConfig.create(
            50, // topK
            0.1f, // minSimilarityScore
            RetrievalConfig.TaskType.RETRIEVAL_QUERY
        )
    )

val response = retrievalAndInferenceChain!!.invoke(retrievalRequest).get().text

With that the language model will process the prompt. During processing it will refer to the embeddings in your vector store to provide what it thinks is the most accurate answer.

As you would expect, what you ask the model will result in different responses. As the embeddings contain a range of simon says responses, it’s likely you will get a good response!

What about the cases where you recieve an unexpected result? At this point you need to go back and fine tune the parameters of your objects from the previous section.

Let’s do that in the next section.

Fine tuning your RAG Powered Language Model

If there’s one thing we’ve learned over the years from Generative AI, it’s that it isn’t an exact science.

You’ve no doubt seen stories where language models have “misbehaved” and produced a less than desired result, causing embarassment and reputational damage to companies.

This is the result of not enough testing being performed to ensure the responses from a model are expected. Testing not only helps avoid embarassment, it can also help to experiment with your language model output so it works even better!

Let’s take a look at what levers we can adjust to fine tune our RAG powered language model. Starting with LLmInferenceOptions.Builder:

LlmInferenceOptions.builder()
    .setModelPath(GemmaModelPath)
    .setPreferredBackend(LlmInference.Backend.CPU) // Change to GPU if you have a GPU powered device.
    .setMaxTokens(1200)
    .build()

The first parameter that can be changed is the setMaxTokens() parameter. This is the amount of “input” or “output” the language model can handle. The larger the value, the more data the language model can handle at once. Resulting in better answers.

In our example, this means the model can handle a text input and generate an output that consists of 1200 tokens. If we wanted to handle less text and also generate a smaller response, we could set MaxTokens to a smaller value.

Be careful with this value, as you could find your model unexpectedly handling more tokens that it expects. Causing an app crash.

Let’s move onto LlmInferenceSessionOptions.Builder():

llmInferenceSessionOptions = LlmInferenceSessionOptions.builder()
    .setTemperature(0.6f)
    .setTopK(5000)
    .setTopP(1f)
    .build()

Here, you can set a couple of different parameters. .setTemperature(), .setTopK(), and .setTopP(). Let’s dive them into each of them.

.setTemperature() can be thought of how “random” the responses from the language model can be. The lower the value, the more “predictable” the responses from the model can be. The higher the value the more “creative” the responses will be, leading to more unexpected responses.

For this example it’s set to 0.6, meaning the model will provide semi-creative, but not unimaginable responses.

The temperature is a good value to experiment with, as you may find different values provide better responses depending on your use case. In a game of Simons Says, some creativity is welcome!

.setTopK() is a way of saying “only consider the top K results to return the user”. Language models whilst processing a prompt generate a number of responses, potentially thousands!

Each of these responses are given a probability as to how likely they are to be the right answer. To limit the amount of responses to consider, the topK value can be set to help focus the model. If you’re happy with less probabable responses being considered, you would want to set this value high.

Similar to the temperarture property, this is a good property to experiment with. You may find the model works better with less or more responses to consider, depending on your needs.

For a game of Simon Says, we want the model to be thinking about alot of different responses to keep the game fresh. So 5000 seems like a good value.

.setTopP() builds upon the limit set by topK by saying “only consider results that have a probability of P”. As mentioned earlier, language models assign a probablility to each response it generates as to how likely it can be. Setting topP means the model can easily discard any responses that don’t have the minimum probability needed.

To show an example, if the model had a topP set to 0.4 and was considering the following responses:

Simons Says clap 5 times! = 0.7 // This response has a probablility of 0.7 of being correct

Simons Says jump up and down! = 0.5

Find a car and drive it. = 0.3

The first two responses would be considered because the probability is higher than 0.4. The response about the car would be discarded, as it only has a probability of 0.3.

Similar to temperature and topK, topP allows you to define how creative your language model can be. If you want to consider less probable responses to prompts, setting a low P value will help.

In our example, it’s set to 1.0. That’s because we want the model to be absolutely sure about its responses. It’s playing a childrens game after all!

Experimenting with these values will generate very different results from your language models. Try them out and see what happens!

Where To Go Next?

I hope you’ve enjoyed this walkthrough of how to add a RAG powered language model to your Android App! If you’re looking to learn more about RAG and Android, here are a few links I recommend:

Simons Says App: Clone and run the sample code for this blog post to see it in action. It shows how to setup the RAG pipeline with Gemma using Android architecture best practices.
MediaPipe RAG : Check out the RAG section on the MediaPipe Google Developer docs Highly recommended reading.
Setting Temperature, TopK, and TopP in LLMs : Learn more about how setting the Temperature, TopK, and TopP values can help control the results from language models. Another highly recommended article.

Multiplatform Development for Apple Devices

Darryl Bayliss — Tue, 28 Jan 2025 06:00:00 +0000

In June 2023 as Apple announced the Vision Pro I had an idea that could work well on the headset. A collection of looped videos playing alongside your day to day usage.

I already had an app that could do this called Christmas Chill, something I built when the 1st Apple TV supporting an App Store was made available. It features a collection of looped videos you can use as a festive backdrop.

Each year over Winter I spend a couple of days to improve it, adding new content and improving the codebase. One of the larger changes made to the project came in December 2023, when the UI was migrated from UIKit and Storyboards to SwiftUI.

Well, mostly migrated. I needed an AVPlayer backed view wrapped in a UIViewRepresentable. A great API providing interoperability between UIKit and SwiftUI, if you’re ever in need.

I had been somewhat hesitant to migrate sooner as I had a good understanding of Declarative UI and its concepts from other work with React and Jetpack Compose.

That changed for me with Apple’s introduction of the Apple Vision Pro and its support for SwiftUI. Christmas Chill has been a nice project to keep my Apple Dev knowledge upto date, and I was keen to gain experience expanding the app to different devices.

Once the 2023 migration to SwiftUI for Christmas Chill was complete I set out in 2024 to add support for the Vision Pro. Below is how I went about it and what I recommend if you were trying to do the same for your own apps.

Adding A New Platform

To start, the project needs to be able to build for Vision Pro as a destination, doing this is surprisingly easy! Inside Xcode, select the .xcodeproj file and under the Supported Destinations dropdown, click the plus button.

A dropdown of all the available Apple platforms appears. Hover over the desired platform to add as a destination, in this case Apple Vision, and then click Apple Vision in the newly appearing section.

A small popup will appear to inform you of changes Xcode needs to make to the target. Click Enable.

Next, build the app using the visionOS Simulator. If you have a Vision Pro to hand you can find instructions on how to install it onto your device here.

During compilation, it’s likely Xcode will find compiler errors and / or the app will crash. This is expected and a practice in being patient. From this point you need to fix the errors in your project until the app compiles and no longer crashes.

In my case this took around 30 minutes, partially thanks to doing the hard work of migrating the app from UIKit to SwiftUI previously!

Conditional Compilation Blocks

SwiftUI at its core is a multiplatform framework, meaning just by compiling SwiftUI code for a different platform it will alter its appearance. Taking into account platform styling and various methods of interaction.

Whilst this helps to make quick progress during development, you may want more control over how the app appears and take advantage of each individual platforms strengths. A good example is the Vision Pro’s Immersion capabilities, SwiftUI provides an API for this via ImmersiveSpace, an API only available for visionOS.

If you tried to use this API whilst compiling the project for Apple TV, Xcode will throw an error informing this API is not available.

So what is the solution to avoid this situation? The answer comes from using Conditional Compilation Blocks. Compilation Blocks are sections of code providing instructions for when the compiler should compile code within the block.

Whilst they support a variety of conditions, the most useful for our needs is detecting which platform the code is being compiled for. You can do this with just a few lines of code:

var body: some Scene {
    #if os(tvOS)
        WindowGroup {
            HStack {
                Text("I am running on tvOS!")
            }
        }
    #elseif os(visionOS)
        ImmersiveSpace(id: "MyImmersiveSpace") {

        }
    #endif
}

A nice feature Xcode does to support conditional compilation blocks is make clear what code will compile depending on the platform selected for compilation. It will also slightly fade out code that won’t be compiled.

Dependency Injection via Build Phases

One of the more useful tricks I’ve found is using the Compile Sources and Copy Bundle Resources build phases as a form of dependency injection. These are processes run when the app is built and can be found under the Build Phases tab in the Xcode Project.

Compile Sources does the heavy lifting of compiling your source code into machine code. Whether its Swift, Objective-C, or even C/C++.

Copy Bundle Resources copies all related resources for the app target into the App Bundle. A container of sorts for all the apps code and resources including images, videos, localisable strings, and more.

These two build phases give alot of flexibility to apps as each new target provides their own build phases, including the two steps above. Whitelabel apps that provide a way for businesses to customise their content use this technique, amongst others.

You may find you want to provide different content for your own apps, depending on what platform they run on. Let’s use these build phases to our advantage and provide two different sources of content to do that.

First, let’s use a Swift Protocol to provide a contract that expects to be fulfilled by a struct or class.

protocol ContentManager {

    var content: [Content] { get }
}

Next, let’s take a look at two implementers of the protocol. Here is the first:

class TargetAppAContentManager : ContentManager {

    var content: [Content] {

            return [
                Content(name: TargetAppAContentIdentifier.videoOneName.rawValue,
                        image: TargetAppAImagePreviewIdentifier.videoOnePreview.rawValue,
                        video: TargetAppAImageVideoIdentifier.videoOneVideo.rawValue),
                Content(name: TargetAppAContentIdentifier.videoTwoName.rawValue,
                        image: TargetAppAImagePreviewIdentifier.videoTwoPreview.rawValue,
                        video: TargetAppAImageVideoIdentifier.videoTwoVideo.rawValue),
                Content(name: TargetAppAContentIdentifier.videoThreeName.rawValue,
                        image: TargetAppAImagePreviewIdentifier.videoThreePreview.rawValue,
                        video: TargetAppAImageVideoIdentifier.videoThreeVideo.rawValue),
        ]

        return contentToShow
    }
}

TargetAppAContentManager is the concrete implementation used for the first app target. It provides an array of Content, which refers to resource names found in the app bundle for the target.

class TargetAppBContentManager : ContentManager {

    var content: [Content] {

        return [
            Content(name: TargetAppBContentIdentifier.videoOneName.rawValue,
                    image: TargetAppBImagePreviewIdentifier.videoOnePreview.rawValue,
                    video: TargetAppBImageVideoIdentifier.videoOneVideo.rawValue),
            Content(name: TargetAppBContentIdentifier.videoTwoName.rawValue,
                    image: TargetAppBImagePreviewIdentifier.videoTwoPreview.rawValue,
                    video: TargetAppBImageVideoIdentifier.videoTwoVideo.rawValue),
            Content(name: TargetAppBContentIdentifier.videoThreeName.rawValue,
                    image: TargetAppBImagePreviewIdentifier.videoThreePreview.rawValue,
                    video: TargetAppBImageVideoIdentifier.videoThreeVideo.rawValue),
        ]
    }
}

Next is TargetAppBContentManager, the concrete implementation used for the second app target. It looks very similar to the first implementation, except for App B the identifiers are different.

With both implementations created, you can now refer to them indirectly in your code by setting the type of the object to ContentManager. Check out the example ViewModel below:

@Observable class VideoListViewModel {

    var contentManager: ContentManager

    init(contentManager: ContentManager) {
        self.contentManager = contentManager
    }
}

The ViewModel expects a type of ContentManager to passed in via its initialiser. The ViewModel can be passed ether type of ContentManager and continue to function as expected. This also means the ViewModel can be reused across both app targets.

The last thing to do is to ensure the correct ContentManager is added to the Compile Sources phase. In this case, App A is passed TargetAppAContentMananger as part of its sources, and App B is passed TargetAppBContentManager.

Adding App Bundle Resources

The last thing left to do is to ensure each app Bundle contain resources with names that match the identifiers used by the app. The easy way is to check the Copy Bundle Resources build phase of each app target and ensure the resources are referred to by the content manager. If not then drag them from your Xcode project into the copy resources phase.

This takes a little bit of time and care to test, as you don’t recieve a compile time error if a resource being referred to isn’t available in the bundle. During runtime you will get a crash!

A good way to automate the check is to write a unit test to confirm all resources being referred to by ContentManager are stored in the bundle. If the test fails when run then you know there is a missing resource in the bundle.

Where To Go Next?

If you’ve got this far you should have a good idea of how to bring your app to other Apple Platforms.

To round off this post, I will leave you with a couple of tips and resources I recommend:

If adding Apple Vision support to an existing app, first migrate as much of your code from UIKit to SwiftUI as possible. Having seen the speed of an existing app working on Vision Pro when migrated to SwiftUI, it is a useful to be able to rely on.
Read through Apple’s guidance on bring existing apps to visionOS. It provides useful tips and suggestions on how to do it and how to take advantage of visionOS features.
If you’re thinking of starting a new multiplatform app yourself there is a Multiplatform tab available in Xcode, providing a number of app templates to use. There is also a video from WWDC 2022 on the subject.
If you would like to see examples of apps working across multiple platforms, I recommend checking out my personal apps Christmas Chill and Ocean Chill. These are two apps working across tvOS and Vision Pro, built from a single codebase. (tvOS support for Ocean Chill coming soon!)

Getting Setup With Xcode Cloud

Darryl Bayliss — Tue, 20 Aug 2024 05:00:00 +0000

Xcode Cloud is a Continuous Integration and Continuous Delivery (CI / CD) platform provided by Apple. It automates away the hard work of building your apps, testing them, signing them, and pushing them to a destination.

Why use Xcode Cloud I hear you say? If you’ve been through the process of manually archiving your app, then submitting it to iTunes Connect, you’ll know these steps take time.

With Xcode Cloud, Apple makes it easy to automate this with a few button clicks through Xcode. Let’s take a look at how to do this.

Setting Up Xcode Cloud

In order to use Xcode Cloud, you need to have a few things setup. The main requirements are:

Enrollment in the Apple Developer Program
An app record on App Store Connect
Your Developer Apple ID added to Xcode

Your project also needs to be stored in a git repository. Xcode Cloud makes heavy use of git as a trigger to know when to start running.

A complete list of requirements is available if you find yourself needing more.

Once you’re setup, it’s time to setup a workflow. Along the top of Xcode, click the git branch so a popup appears. You’ll see an option called Create Workflow.

Click the option and Xcode will show a new window, asking which product to create a workflow for.

Select an app and click Next. You’ll be taken to the next screen to customise your workflow.

Creating a Workflow

The workflow screen contains a few different sections. First, it shows the General section where you can provide infomation about your workflow.

Here, you can give your workflow a name and a description on what it does. This is useful for yourself as a solo developer, or as a team where you have multiple engineers who need to use the same workflow.

You can also restrict the workflow from being edited in this section, ensuring workflows aren’t tampered with by other team members without the right permissions.

The next section is the Environment section. Here, you can select what version of Xcode and macOS your workflow will use.

Xcode Cloud provides a variety of Xcode versions, including beta versions. This is handy if you want to create multiple workflows building and testing different configurations.

The environment section also provides the option to clean the workflow before each build. With the option unchecked, each workflow stores the derived data and other infomation from each build to speed up subsequent builds.

Whilst this can be a good thing, you may want to avoid this depending on your needs. For example, delivering a build to TestFlight for external testing requires a clean build.

Finally, you can provide environment variables to the workflow. These can be picked up by custom build scripts used by your app to extend workflows. If a variable value needs to be kept secret, you can check the secret tickbox and the value will be hidden.

With the infomation for the workflow setup and the environment configured, the next step is to create triggers that cause the workflow to run.

Start Conditions

The next section is called Start Conditions. Here, you can tell the workflow what conditions should trigger it to run. By default, Xcode sets the start condition to be any change to any file on any branch.

It’s recommended to tweak this as your workflow will run for every commit. One option is to constrain the start condition to run for a certain branch. First, select the Custom Branches radio button, then begin to type the name of the branch. Xcode will search your git branches and suggest branches that match your search.

You can also choose to trigger workflows based on changes occurring to a specific file or folder, including specific types of files. Multiple files triggers can be added, giving you the ability to build complex trigger rules for file changes.

If you need a different trigger, you can create different start triggers by clicking the + next to the start condition. Xcode provides different triggers for a number of situations.

These range from changes to git tags, pull request changes, a scheduler for a specific branch, and finally a manually started workflow.

One common option across most of the triggers is the ability to Auto Cancel Builds. This is useful if your trigger is likely to start the workflow multiple times. Instead of waiting for each build to finish, Xcode Cloud cancels the older builds and prioritises the latest build.

This is so useful, it’s enabled by default!

When you’re happy with your workflow start triggers, you’re ready to move on and set up actions for the workflow to perform.

Workflow Actions

As part of any good CI / CD platform, performing actions regularly and repetitively is good practice to ensure your app is builds successfully and you can catch issues quickly.

Xcode Cloud is no exception, and allows you to choose from a selction of Actions. By default, your workflow includes a Build action to build your chosen app scheme.

If there are issues with the scheme, Xcode Cloud provides a helpful warning so you can address it.

Similar to start triggers, you can add more build actions that are run as part of the workflow. These include running tests, running the analyzer, and archiving your app before distributing it to TestFlight or the App Store.

The test and analyse build actions contain a Requirement setting. This setting decides whether the build action must pass for the workflow to continue. For example, if you want the workflow to fail if one of your tests fail then you can enforce this requirement.

Interestingly, the test build action doesn’t seem to support visionOS at present.

You can configure as many build actions as needed for your workflow. You could for example, run the tests for each scheme in your project, then build, archive, and distribute each app in one workflow.

Once you are happy with your build action setup, you can move to the final part of the workflow. Setting up post actions.

Post Actions

Post Actions allow you to perform a couple of actions before the workflow is complete. These include distributing new builds to TestFlight users, notarizing macOS apps, and notifying others of the workflow result through Slack or email.

The Notify post action gives you a degree of control over what messages are sent. You can choose to recieve all Build Success messages, to be notified when the workflow is fixed from a previous failure, or not to be notified at all!

Similar options for Build Failure messages are available. You can choose to recieve all failure messages, breaking workflow messages only, or no failure messages at all.

Saving Your Workflow

Once you are happy with your Workflow. Click the Save button at the bottom right of the window.

Xcode Cloud will check the workflow and let you know if there are issues that must be addressed. If all is well, the workflow window disappears and takes you back to Xcode.

Click on the git branch along the top of Xcode again, and this time click Manage Workflows. A new window will appear, showing the workflow you created for the app.

If you want to edit the workflow, select the workflow and click the gear button in the bottom left corner. You can also click the - button whilst the workflow is selected to delete it.

Running Shell Scripts In Xcode Cloud

Xcode Cloud provides support for running shell scripts at specific parts of the workflow. This is useful if you need to perform more advanced configuration for your workflow, and allows your workflow to be customised even further.

Exploring this topic is beyond the scope of this post, however you can find out more in the Developer Documentation.

Viewing Your Workflow In Xcode And App Store Connect

You can view the results of your workflow in Xcode by clicking the Report Navigator and clicking the Cloud tab. Each workflow will be shown, including the results from each build.

You can also view and interact with your workflows in App Store Connect by navigating to your App Store listing and selecting the Xcode Cloud tab along the top of the listing.

Costs

Xcode Cloud comes included in every Apple Developer Program Membership. Each membership comes with 25 hours of compute time per month. If you go over then you can choose from a variety of subscription options to recieve more time.

Finally

If you enjoyed this blog post, consider dropping me a couple of $$$ over on Kofi.

You can also follow me on Social Media to hear more about software engineering by clicking the links below.

X / Twitter: https://x.com/darryl_bayliss

LinkedIn: https://www.linkedin.com/in/darrylbayliss/

Mastodon: https://mastodon.social/@darryl_bayliss

GitHub: https://github.com/DarrylBayliss

Which Map Transformation Should I Use?

Darryl Bayliss — Thu, 04 Jul 2024 05:00:00 +0000

Map transformation functions find common usage in Android development. They are part of the Kotlin Standard Library, a library built by JetBrains to provide standard functionality across Kotlin codebases.

Inside the Standard Library is a package called kotlin.collections, containing the building blocks for different collections. These include Lists, Maps, and Sets.

The collections package also contain the map transformation functions. These functions take the contents of a collection and transform them into another collection containing the transformed state.

Let’s take a look at some examples.

The Map Transformation

The first transformation is the map() function.

  val numbersList = listOf(1, 2, 3)
  numbersList.map { it + 1 }.also(::println) // listOf(2, 3, 4)

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)
  numbersMap.map { it.value + 1 }.also(::println) // listOf(2, 3, 4)

  val numbersSet = setOf(1, 2, 3)
  numbersSet.map { it + 1 }.also(::println) // listOf(2, 3, 4)

This function iterates over a collection and applies the transformation to each value within a lambda, before returning a new collection containing the transformed values.

map() is available across different types of collections. The reason for this is because map() is an extension function on the Iterable interface. Most collections implement Iterable, meaning they can make use of the map function.

Map collection types are different. They don’t implement Iterable, instead they possess a separate extension method to provide a map function that iterates over each entry and returns a List of results.

Map Transformations and Null Values

Map transformations come in different forms. Another useful transformation is the mapNotNull() function.

  val numbersList = listOf(1, 2, 3)
  numbersList.mapNotNull { if (it + 1 == 3) null else it + 1 }.also(::println) // listOf(2, 4)

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)

  numbersMap.mapNotNull { if (it.value + 1 == 3) null else it.value + 1 }.also(::println) // listOf(2, 4)

  val numbersSet = setOf(1, 2, 3)
  numbersSet.mapNotNull { if (it + 1 == 3) null else it + 1 }.also(::println) // listOf(2, 4)

This function acts both as a transformation function and a filter for null values. If the transformation inside the lambda results in null then the value is not added to the new list.

mapNotNull() is available to collections implementing the Iterable interface.

Map types have their own extension function to provide similar functionality. Returning a list of results.

Acquiring an Index with Map Transformations

If you need to know the location of the value within the collection being transformed you can use mapIndexed().

  val numbersList = listOf(1, 2, 3)
  numbersList.mapIndexed { index, number -> number + index + 1 }.also(::println) // listOf(2, 4, 6)

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)

  numbersMap.asIterable().mapIndexed { index, entry -> entry.value + index + 1 }.also(::println) // listOf(2, 4, 6)

  val numbersSet = setOf(1, 2, 3)
  numbersSet.mapIndexed { index, number -> number + index + 1 }.also(::println) // listOf(2, 4, 6)

Here the location of the value (the index) within the collection is passed alongside the value being transformed. mapIndexed() is available to collections implementing the Iterable interface.

Map types don’t have a mapIndexed() extension function. What you can do though is use the asIterable() extension to wrap the Map inside an Iterable instance. Then you can use mapIndexed() without issue.

If you need to check for null values and also require an index you can also use mapIndexedNotNull().

  val numbersList = listOf(1, 2, 3)
  numbersList.mapIndexedNotNull { index, number -> if (number + 1 == 3) null else number + index + 1 }.also(::println) // listOf(2, 6)

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)

  numbersMap.asIterable().mapIndexedNotNull { index, entry -> if (entry.value + 1 == 3) null else entry.value + index + 1 }.also(::println) // listOf(2, 6)

  val numbersSet = setOf(1, 2, 3)
  numbersSet.mapIndexedNotNull { index, number -> if (number + 1 == 3) null else number + index + 1 }.also(::println) //listOf(2, 6)

mapIndexedNotNull() works similarly to mapNotNull(). It filters away null values within the transformation lambda whilst also passing in the index for the value from the collection. Like other map transformations it exists on all types implementing Iterable.

Map types can use the asIterable() function to gain access to mapIndexedNotNull().

Other Transformations for Map Types

Map types work differently than other collections due to not implementing the Collection or Iterable interfaces. Because of this they have a few of their own transformation functions not available to other types. The first is called mapKeys().

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)

  numbersMap.mapKeys { entry -> entry.key.capitalize() }.also(::println) // mapOf("One" to 1, "Two" to 2, "Three" to 3)

mapKeys() transforms each key within the map by passing through each Entry of the map. Once all the transformations are complete they are applied to the Map.

The second function is called mapValues().

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)

  numbersMap.mapValues { entry -> entry.value + 1 }.also(::println) // mapOf("one" to 2, "two" to 3, "three" to 4)

mapValues() works in a similar way. It passes through each Entry of the map and transforms each value. Once all the transformations are complete they are applied to the map.

Passing Map Transformations to a Destination

If you want to pass applied transformations to a different collection other than the source there are a few functions to help.

  val numbersList = listOf(1, 2, 3)
  val numbersDestinationSet = mutableSetOf<Int>()
  numbersList.mapTo(numbersDestinationSet) { it + 1 }
  println(numbersDestinationList) // setOf(2, 3, 4)

  val numbersMap = mapOf("one" to 1, 
                         "two" to 2, 
                         "three" to 3)
  val numbersDestinationList2 = mutableListOf<Int>()
  numbersList.mapTo(numbersDestinationList2) { it + 1 }
  println(numbersDestinationList2) // listOf(2, 3, 4)

  val numbersSet = setOf(1, 2, 3)
  val numbersDestinationList = mutableListOf<Int>()
  numbersSet.mapTo(numbersDestinationList) { it + 1 }
  println(numbersDestinationList) // listOf(2, 3, 4)

The mapTo functions work similarly to map(). The difference is they write the transformations to the passed in collection. The collection being written doesn’t have to be the same type as the source collection. Useful if you have a usecase where a different collection would be more optimal.

Map types can’t use mapTo(). This is because mapTo() expects to write to a MutableCollection, which Map types don’t inherit from.

There is a MutableMap type, however because it doesn’t inherit from MutableCollection there is no mapTo() extension available.

More Resources

If you’d like to learn more about Kotlin’s transformation methods. I highly recommend this page on collection transformations from the Kotlin language documentation.

As well as covering map transformations it also covers other methods like zipping, association and flattening. These topics are beyond the scope of this post however are nevertheless useful to understand.

Playing Simon Says with Gemma-2b and MediaPipe

Darryl Bayliss — Sat, 16 Mar 2024 05:00:00 +0000

A couple of weeks ago I attended Google’s Gemma Developer Day. A day dedicated to Google presenting the capabilities, portability and openness of their newest LLM models called Gemma.

These models present an exciting new phase for Generative AI. Models small enough to be deployed on local devices and can still provide an engaging and helpful experience.

It’s been a while since I looked at Mobile AI and went away from the day feeling inspired. I decided to see what happens when I tried these models in an Android App.

The App

I wanted to give Gemma an easy yet novel challenge. Playing a game of Simon Says.

The rules are simple. One player gets to be Simon. Their role is to give the other players tasks to perform. The player playing as Simon has to say “Simon says” before they give the task.

I created an app with three screens. An entry screen, an instructions screen, and a chat screen where Gemma can communicate and give tasks.

To speed up building the chat screen I found this blog post from Meyta Taliti extremely helpful.

With the screens created my next task was to integrate Gemma. For this I relied on a set of tools I learnt about called MediaPipe.

MediaPipe

MediaPipe is a collection of tools with the goal of simplifying integrating AI models into apps on Android, iOS, and the web. With MediaPipe you have alot of choice depending on your needs.

If you want to get started quickly MediaPipe provides an API called Tasks for you to call out to AI models. These API’s are split into different areas such as Vision, Text & Audio.

MediaPipe also provides a collection of pretrained models to embed within your apps. Again, useful for getting starting quickly.

If you need something more custom and don’t want to create a model from scratch MediaPipe provides a tool called Model Maker. Model Maker uses a process called Transfer Learning to retrain an existing machine learning model and provide it with new data.

The benefit of this approach is it saves time and requires less training data to create a new model.

Model Maker can also reduce the size of the created model through this process. Do note that this process causes the model to “forget” some of its existing knowledge.

The final tool from MediaPipe is MediaPipe Studio , a web application to evaluate and tweak your custom models. Useful if you want to benchmark your models and understand how well they work prior to deployment.

For our needs we’re going to leverage the LLM Interfence API, a new API for MediaPipe. This allows us to communicate with Gemma and recieve a response.

Putting MediaPipe to Work

To use MediaPipe you first need to add it as a gradle dependency to the app:

implementation ("com.google.mediapipe:tasks-genai:0.10.11")

Next, you create an instance of LlmInference. This is the object you use to communicate with Gemma:

val llmInference = LlmInference.createFromOptions(
            context,
            LlmInference.LlmInferenceOptions.builder()
                .setModelPath("/data/local/tmp/llm/gemma-2b-it-cpu-int8.bin")
                .build()
        )

It’s important to note the path set using .setModelPath. This is the where the Gemma model resides on the device. It’s also important the Gemma model used is the gemma-2b versions. The 7b versions are not yet supported by MediaPipe, more on what this all means later. For now let’s download the model.

You can download Gemma from Kaggle. A website dedicated to Data Scientists and Machine Learning. You need to create an account and accept the Terms & Conditions of use before you can download the models. You can find the Gemma page here.

If you’re following along to this post remember to only download the gemma-2b-it-cpu versions of the model, under the TensorFlow Lite tab. You’re on your own if you try the gemma-2b-it-gpu versions.

Once the model is downloaded. Use the Device Explorer in Android Studio to import the model to the path set in .setModelPath. If you’ve changed the path or the model name then make sure to update the path name.

Once the model is imported you can begin to pass prompts into Gemma using the .generateResponse method. Here is an example of the prompt I pass to Gemma to play Simon Says:

private const val SimonSaysPrompt = """
    You are a Simon in a game of Simon Says. Your objective is to ask the player to perform tasks.

    For every task you give, you must prefix it with the words "Simon says".

    You must not ask the player to do anything that is dangerous, unethical or unlawful.

    Do not try to communicate with the player. Only ask the player to perform tasks.
"""

val gemmaResponse = llmInference.generateResponse(SimonSaysPrompt)

If you’ve used LLMs before and have a basic understanding of Prompt Engineering this should look familar. To err on the side of caution I’ve included precautionary instructions in the prompt. We don’t want Simon asking the user to do anything questionable!

If you try to run this on a device a couple of things may happen.

The app may take a short while to respond and eventually provide a response.
The app may crash. Looking in Logcat you’ll see messages about MediaPipe being unable to find the model. Did you set the right model path?
The app may crash. If you look in Logcat you can see alot of native code logging and infomation about memory being recycled.

My experience fell into the second and third category. Your own experiences may vary if you’ve setup everything correctly and using a high spec physical device.

If you don’t have ether of these things. There’s another option, increasing the amount of RAM available through the Emulator.

Creating an Android Emulator with Increased RAM

Increasing the amount of RAM available usually helps in a memory intensive environment so why would a memory hungry LLM be any different? To do this I customised the amount of RAM my Android emulator used.

If you have an existing emulator you may notice the RAM field is disabled. You can still update the amount of RAM it has available by clicking on the three dots to the right of it in the Device Manager.

Click Show on Disk and then open up the config.ini and hardware-qemu.ini files in a text editor. Change the values of hw.ramSize in each file. Thanks goes to this Stack Overflow question for giving me the answer on how to do it.

Alternatively you can create a custom emulator by going to the Device Manager in Android Studio, clicking Create Virtual Device and then New Hardware Profile. As part of the options to customise you can select the amount of RAM.

I found 8GB of RAM to work relatively well. I also tried my luck with 22GB of RAM. It performs marginally better in terms of speed, although not as much as I expected.

I suspect there is a bottleneck somewhere when Gemma is loaded into memory, as the rest of the emulator runs fluidly. Perhaps an improvement somewhere that can be made.

Gemma 2b & Gemma 7b

The Gemma models compatible with MediaPipe are the gemma-2b versions. The 2b stands for 2 billion parameters. The amount of parameters working together to make the model work.

These are the values set within the model during training to provide the connections and inferences between each other when you ask Gemma a question.

There is also a gemma-7b collection, which use 7 billion parameters. These are not supported by MediaPipe however. Maybe one day!

If you’re interested in understanding more about parameters when it comes to LLMs I recommend this page.

Having a 2 billion parameter model being loaded and running on a mobile device is an impressive achievement. How well does it work though? Let’s find out.

gemma-2b-it-cpu-int4

The gemma-2b-it-cpu-int4 is a 4 bit LLM. This means each parameter used by the model has a memory size of 4 bits. The benefit here is the total size of the model is smaller, however the reduced memory size for each parameter means the accuracy and quality of the model are also affected.

So how does gemma-2b-it-cpu-int4 perform? Not so great to be honest. Here are a few screenshots of my attempts to play Simon Says using the prompt above and asking it general questions.

The responses were unexpected and it was frustrating to get the model to do anything resembling a game of Simon Says. It would veer off into a different topic and hallucinated inaccurate infomation.

Hallucinations are a phoenema where LLMs speak falsehoods and untruthful things as if they are fact. Take the example above, it’s not true you can drive to Mars in 60 minutes at 60mph. Not yet anyway. 😃

There was also a lack of context awareness. Meaning it couldn’t remember something I mentioned earlier in a conversation. This is likely due to the constrainted size of the model.

After a while I gave up on this model and decided to try the larger 8 bit sized model.

gemma-2b-it-cpu-int8

The gemma-2b-it-cpu-int8 is an 8 bit LLM. Its larger in size to its 4 bit sibling. Meaning it can be more accurate and provide better quality answers. So what was the outcome here?

This model was able to grasp the idea of Simon Says, immediately assuming the role of Simon. Unfortunately it too suffered from a lack of context awareness.

To counter this I needed to reprompt the model everytime with the rules of Simon Says and combine it with another prompt to ask it to provide a task.

The task prompts are randomly picked from a list to pass into Gemma, giving some variety in the tasks being asked.

Here is an example of what is happening below:

private const val SimonSaysPrompt = """
    You are a Simon in a game of Simon Says. Your objective is to ask the player to perform tasks.

    For every task you give, you must prefix it with the words "Simon says".

    You must not ask the player to do anything that is dangerous, unethical or unlawful.

    Do not try to communicate with the player. Only ask the player to perform tasks.
"""

private const val MovePrompt = SimonSaysPrompt + """
    Give the player a task related to moving to a different position.
"""

private const val SingASongPrompt = SimonSaysPrompt + """
    Ask the player to sing a song of their choice.
"""

private const val TakePhotoPrompt = SimonSaysPrompt + """
    Give the player a task to take a photo of an object.
"""

private val prompts = listOf(
    MovePrompt,
    SingASongPrompt,
    TakePhotoPrompt
)

val prompt = prompts.random()
val response = llmInference.generateResponse(prompt)

It does ocassionally throw a curve ball response that seems out of character. I’m putting this down to the size of the model. It’s also worth considering this is only v1.

Once the prompts were set in stone I found it was useful to rely on the prompts only and not take the user input into consideration. Because the model lacks context awareness the user input causes it to stop playing Simon Says and instead respond to the input.

Adding this bit of trickery wasn’t a satisfying outcome, but one needed to keep Gemma playing Simon Says.

Impressions and Thoughts

So can Gemma play Simon Says on an Android device? I’m going to say “kind of, with help”.

I would like to see the 4 bit version of Gemma 2b responding more intuitively. Making Gemma 2b context aware to avoid the need to reprompt it for every request and being careful with user input would help too.

For simple requests needing only a single prompt. I can see Gemma 2b being able to comfortably handle these tasks.

Its also worth bearing in mind these are v1 of the models. The fact they run and work on a mobile operating system is an impressive achievement!

The Future of on Device LLMs

What about the future of LLMs on mobile devices? There’s two barriers I see. Hardware limitations and practical use cases.

I think we’re at a point where only high end devices can effectively run these models. Devices that come to mind are the Pixel 7 or Pixel 8 series of phones with their Tensor G chips, and Apples iPhone with their Neural Engine chip.

We need to see these kind of specifications filtering through to mid-range phones.

Interesting ideas could come from on device LLMs tapping into Retrieval Augmented Generation. A technique for LLMs to communicate with external data sources to retrieve additional context when providing answers. This could be an effective way to boost performance.

The second barrier is finding practical use cases. I think these are limited whilst devices can communicate with more powerful LLMs over the internet. GPT-4 from OpenAI for instance is rumoured to support over a trillion parameters!

There could come a time though where the cost of deploying these models on mobile devices becomes cheaper than hosting them in the cloud. Since cost cutting is all the rage these days I can see this being a viable use case.

There’s also the privacy benefits of having your own personal LLM, with no infomation leaving the confines of your device. A useful benefit that will appeal to privacy conscious app users.

My bet is we’re still a few years away from LLMs being regularly deployed on device.

Useful Resources

If you’re keen to try Gemma for yourself on a mobile device here are some resources to help:

Gemma : The offical Gemma website contains a wealth of infomation including benchmarks, quick start guides, and infomation on Googles approach to responsible Generative AI development.

MediaPipe : MediaPipe has its own Google Developer section where you can learn more about it and how to use it. Highly recommended reading.

Google Developer Group Discord : The Google Developer Group Discord has dedicated channels to Generative AI. Check out the #gemma, #gemini and #ml channels to chat with like minded people.

Simons Says App: Clone and run the sample code for this blog post to see it in action. It also includes usage of the Image Classification Task from MediaPipe. Setup instructions are in the README.

Footnotes

Updated 23/03/24 to mention calling the LLM inference from an IO thread

It occurred to me after writing this post that calling out to gemma is a read / write operation on a file. Moving the .generateResponse() method out to an IO thread will avoid the immense jank when gemma is loaded into memory:

    suspend fun sendMessage(): String {
        return withContext(Dispatchers.IO) {
            val prompt = prompts.random()
            llmInference.generateResponse(prompt)
        }
    }

Interacting with Apple Maps Through URL Parameters

Darryl Bayliss — Sun, 18 Feb 2024 06:00:00 +0000

I recently made a change to EVBuddy to update what happens when a user needs directions via Apple Maps.

From the outside it’s a simple action. The website takes the user to the Maps App.

The interaction between EVBuddy and the Maps App on MacOS is possible because Maps provides a URL Scheme. A way for Maps to understand an URL and show the correct content.

If you opened a web link before and been directed to an App, you’ve used the App’s URL Scheme!

The Apple Maps URL Scheme

So how does the Maps URL Scheme work? Previously, I used the following URL:

const appleMapsUrl = "http://maps.apple.com/?q="

The important part of the URL here is q=. This treats what comes after = as a query, meaning Maps acts like a search engine.

Type in Cinemas, Beach or Italian Restaurants and Maps will open and search for a matching result.

You can try this for yourself if you use an iOS, iPadOS or MacOS device with Maps installed. Type http://maps.apple.com/?q= into your browser and after = search for something.

Adding a Destination

For EVBuddy, I don’t want to search, I want to provide a destination for people to drive to and charge their car. How can we do that?

Fortunately, Apple provide a page with all the accepted parameters to use with Maps. You can find the page here.

Looking at the page, Maps supports a parameter called daddr. This stands for Destination Address, which we can pass the Postcode of the charging point as our destination.

Our URL now looks like this:

const appleMapsUrl = "http://maps.apple.com/?daddr="

Choosing the Transport Type

Next, it would be good if the directions are calculated by car. Maps gives different directions depending on the type of transport.

To do that, there’s another parameter called dirflg we can use. The value of a car for this parameter is d.

If you have an idea what this parameter name means let me know! My best guess is it was used for something else before being changed to its current purpose. 😃

Adding this parameter to the URL, we can see it building up actions as we ask Maps to find directions for a car to the destination address:

const appleMapsUrl = "http://maps.apple.com/?dirflg=d&daddr="

Selecting the Map Type

Finally, it would be good if a certain type of map is shown. Maps can show different types of map views depending on the users preference.

In our case we want to show the standard map view so directions are presented clearly.

Looking at the Maps URL parameters page, there’s a parameter called t (maybe it stands for type?) we can use to set the map type. The value to show the standard map is m.

Adding this parameter, we see the completed URL asking Maps to show the standard map and find directions for a car to the destination address:

// Show Apple Maps using the standard view (t=m), 
// Using car as the mode of transport (dirflg=d) 
// And appending a destination address (daddr=)
const appleMapsUrl = "http://maps.apple.com/?t=m&dirflg=d&daddr="

One of the nice things about Maps is we don’t need to provide a starting address. Maps automatically uses the device location if it has perrmission.

Building EV Buddy

Darryl Bayliss — Mon, 18 Sep 2023 16:54:16 +0000

In the past couple of years I've become interested in the cloud. I've spent a good chunk of my career working in mobile development, and if theres one trend I don't think is going away soon is apps powered by a platform. As any good engineer would do, I was keen to learn more.

To start, I put myself through AWS certifications. I received AWS Cloud Practitioner Certification last year (passed first time!) and earlier this year passed my Associate Solutions Architect Certification (passed on the 2nd attempt after a washout of a 1st attempt).

Official pieces of paper saying you can do things get a bad rep, I've met a few people who consider them worthless. I can understand this way of thinking as alot of these people have years of hard earned experience.

I can also understand why employers and engineers like them. To me, they're an extra signal to employers and organisations that you have experience in the areas you claim to in your CV and done the hard work to prove it.

So certifications are nice, but I also wanted a project to help showcase that knowledge. As part of my learning, I decided to build a full stack application. I'm happy to say v0.0.1 is ready and live, and it's called EV Buddy.

EV Buddy

EV Buddy is an interactive map showing publicly available electric vehicle charging points. You search for charging points using a postcode, and the map displays public charging points nearby. Users can then click on the charging point to see more infomation. For v0.0.1, EV Buddy is limited to searches in the UK.

Each charging point provides infomation such as what charging connectors the point supports, who controls the charger, whether payment is required for usage, and the address of where it is located. Useful things to know if you're planning a journey in your EV.

EV Buddy links directly to other mapping platforms. If you'd like to use Google Maps or Apple Maps to get directions to that charger, it's only a click of a button away.

The data for the UK is requested from the National Charging Registry, a database of public charging points available across the UK. It's free to access and regularly updated, meaning the map also benefits from these updates.

Let's take a look at the stack.

The Technical Stack

EV Buddy uses the following technologies:

Frontend

ReactJS (Frontend Development)
Mapbox GL JS (Mapping Provider)
MUI (Material Design UI Library for React)

Backend

Node.js / Express (Backend Development)
Serverless Framework (Packages code into deployable AWS Lambda functions)

Cloud Infrastructure

AWS S3 (Object Storage)
AWS Lambda (Serverless computing)
AWS Cloudfront (Content Distribution Network for Website)
AWS Route 53 (Domain Registration / DNS Record Management)
AWS Certificate Manager (Website / Endpoint Encryption)

Below is the cloud architectural diagram of how EV Buddy is put together.

Although there was alot of trial and error involved to get it right, the architecture diagram is not very complicated. Below, I've wrote up the more interesting challenges faced, the decisions made for the architecture, and the outcomes from that.

Choosing a Web Framework

The first decision was the choice of tools for frontend development. ReactJS felt a natural choice as alot of the concepts behind Declarative UI frameworks on mobile find their roots within React. It was also a familar tool, as I'd completed a FreeCodeCamp backend course that used React as it's Frontend framework of choice.

I went with using Create React App to setup the initial build. There was little thought here except I wanted to get started quickly and Create React App allows you to do just that.

Now I have more knowledge of the ecosystem I may of considered using Gatsby or Next.js. My understanding is these frameworks provide more features like mobile support and automatic optimisations than the barebones setup you get with Create React App.

Deciding on a Mapping Provider

The next step was to choose a mapping provider. I initially tried Apple Maps Web API, unfortunately they only provide a Javascript based API. Something I wanted to avoid writing too much of within a React App.

There are attempts by people to wrap Apple Maps within React Components. For one reason or another I found these attempts to be lacking for what I needed.

My next attempt was with Google Maps. This was a similar situation to Apple Maps, very little ReactJS support and a few brave developers trying their hand themselves to wrap Google Maps.

After getting setup with an API key and trying out some of these frameworks I found myself running into issues again. The map wouldn't load, there were graphical glitches. Enough trouble for me to decide persuing this further wasn't worthwhile and decide to move on.

The third attempt was to use Mapbox GL. This time I was making progress, I quickly had a map rendering and could move it about with little trouble. The API is again Javascript based but the integration and support from Mapbox to use their API with React made it easy to work with.

There are rough edges to the API that make Mapbox feel lacking. For instance, there's no click method available for an individual marker. There's an issue raised dating from 2019 about it, with people suggesting various workarounds.

The workaround I picked worked, but seems like a poor developer experience to have to rely on a workaround to detect a marker click.

Despite these faults, I'm finding my use of Mapbox GL to be a good decision.

Node.js, Serverless and AWS Lambda

My next decision was to decide on the tools to use for backend development. My FreeCodeCamp course had used Node.js for the backend, so I decided to rely on that experience to code a small server. The purpose of it to calling out to the National Charging Registry API and return the response to the website.

This was an easy exercise. The difficult part came when deploying the server to AWS and balancing costs. I didn't want the hassle of having to manage a server, so initially went with AWS Elastic Beanstalk.

Elastic Beanstalk is a service taking the pain out of managing an environment and does all the hard work for you. You give it the code, and it gets to work deploying everything you need for your environment. It's an impressive product.

The downside of Elastic Beanstalk is it's expensive, even for tiny workloads and deploying iterations. I barely did anything and managed to rack up an AWS bill of about ~$80 in 1 month. No where sustainable in my eyes for a small project.

I decided to see if Lambda functions were cheaper on AWS and could still work with my Node.js server. To my surprise AWS offers a free tier for Lambda functions, allowing 1 Million free requests per month.

This was progress, however I had built a server, not a Lambda function. A quick internet search solved this problem though, as I found a framework called Serverless. Serverless provides a wrapping for Node.js, turning it into a deployable Lambda function.

Serverless also provides tooling to automate deployment of the function via config files, including setting a custom domain and securing the domain with a SSL certificate.

With this I found a solution that is essentially free to run and gives the flexibility to deploy the lambda separately to the website. Deploying via Elastic Beanstalk on the otherhand, meant the website and server were tightly coupled.

Route 53

Once the website and Lambda were in a good state I needed to decide upon a domain name and what DNS registar to use. Fortunately AWS provide their own, called Route 53. Route 53 provides most of the features you would expect from a DNS Registar, ranging from Domain purchase to Adding Records for your domain.

Route 53 also provide additional features like DNS routing policies. These route traffic depending on conditions such geographical location, weighted distribution between destinations and even latency to ensure the request provides the quickest response.

Route 53 also uses the concept of a Hosted Zone, a place where DNS records are stored for your domain. Useful if you have multiple domains.

One of the things I found with hosted zones is the NS records change if you delete and recreate a hosted zone. This can be a problem if you rely on the NS records for operations like DNS validation for an SSL certificate. The related NS records never change for a domain name, but do for a hosted zone.

I figured this out the hard way and only realised the issue after about 6-8 hours of investigation. A mistake I'll aim not to make again!

Once my hosted zone was setup with my domain, I was now ready to use it for my application.

AWS Certificate Manager

I wanted to make sure the website and Lambda function were accessible using HTTPS. To do that I needed a certificate, and to get one I had to use a service called AWS Certificate Manager.

AWS Certificate Manager provides certificates for web domains, including subdomains. Before AWS issues the certificate it requires you to verify yourself as the owner of the domain.

This is easy to do if you can use DNS Validation, a way of allowing AWS to use DNS to verify you as the domain owner using NS records. These records are already setup if you use Route 53, or you can add them to your own DNS Registry.

As mentioned in the previous section, I had deleted and recreated the hosted zone created for evbuddy.io, which meant the NS records AWS were looking for didn't match.

Once I fixed this however the validation happened in a couple of minutes and I had a certificate ready to use.

Setting up the stack to use the certificate was a few clicks in the AWS console, or Click-Ops if you're familar with the term. 😉

S3 & Cloudfront

With a domain and certifcate ready to use, my next step was to store my website on AWS and provide access via my domain. The simplest way to do this was to create an S3 bucket and use it as the origin for a Cloudfront distribution.

S3 is an object based storage product. It can store files in various formats and provides configuration on where these files can be accessed and how long it takes to access them.

The configuration of S3 determines the pricing paid. By default new users of AWS get 12 months free of standard S3 storage with 20,000 GET requests and 2,000 PUT requests.

Outside of that AWS gives generous transfer rates in and out of S3. Transfer rates in are free of charge and transfer rates out are free for the first 100GB per month.

For EV Buddy, I created two S3 buckets. The first one contains the files for the website, whilst the second bucket points to the 1st bucket. The reason for this is both buckets are intended to be used as content points for AWS Cloudfront. Each responds to different URL requests a user may make.

The idea is it makes rerouting requests the user makes to the same website in varying ways simpler. A user querying evbuddy.io can be treated differently to www.evbuddy.io for instance.

With the buckets setup, it was time to turn attention to provide access to the website stored within S3. This is done is via AWS Cloudfront.

Cloudfront is AWS's content distribution network product. It provides a way for developers to distribute their applications, APIs and content across the globe with low latency, using the infrastructure built by AWS. It's designed to be highly available, secure and scalable for your needs.

Configuring cloudfront is simple. You tell it where to fetch the content it needs to deal with, let it know relevant infomation like the domain name to respond, what certificate to use for HTTPS, what the caching policy is and you're done!

Similar to the S3 bucket setup, EV Buddy uses two Cloudfront distributions. Each one handling a different domain request. Both distributions eventually route through to the same S3 Bucket.

One snag I noticed is Cloudfront's aggressive approach to Caching. If I push a change to the website, it wouldn't reflect instantly. This isn't neccessarily a bad thing, as Cloudfront is doing what it's supposed to. It just means I had to be mindful to immediately invalidate the cache once a change has been pushed.

To solve this, I set object metadata on the files stored in S3. This metadata allowed me to set a Cache-Control value for each file, Cloudfront respects this value by checking it before delivering content.

If the value is older than what Cloudfront expects, Cloudfront updates the content from S3 and passes that through to the requester. Magic!

Costs

2023 has been a hard year for most companies and cost cutting is high on the agenda. I thought it was good to keep this in mind and see how well I can optimise costs. Here is a brief look into the costs:

The peak at the left of the graph shows the costs of experimenting with Elastic Beanstalk. These were raising into nearly $80 per month territory, which as mentioned earlier was way beyond my budget.

With the restructuring to use AWS Lambda and relying on free tier pricing for AWS Lambda, S3, and AWS Cloudfront I managed to slash these costs to $1.66 per month as of writing. I expect an element of variance in these costs depending on traffic, but I'm putting a comfortable estimate it won't raise to more than $5 per month.

One outlier cost not shown is $71 for the evbuddy.io domain name. I expect this to be an annual cost for as long as the website is available. .io domain names aren't cheap, but they do add some unique flair to a website URL. :)

In total that translates to a cost of $90.92 per year to run evbuddy.io. The worse case scenario in my mind if costs unexpectedly rise would be $130.00 per year.

Not bad for a website hosted within S3, distributed across the globe and communicating with an external API.

Improvements

Below is a list of improvements I would like to make to EV Buddy in the future.

More countries: EV Buddy so far only supports the UK. I'd like to add support for more countries.

Better Mobile Support: The website is usable on mobile, but has usability problems. The map can get stuck when closing the detail page and the toolbar is minimal.

Improved Connector Detail Page: I'd like to clean up the connector detail page. It is basic and unless you're a EV enthusiast may not be very helpful.

Filtering: I'd like to include filters to the map for users to find chargers based on their preferences. Filters could include charging output, charging networks, or connector types.

Move to TypeScript: The frontend is written entirely in Javascript, but I would like to migrate it to Typescript for the type safety it provides.

Closing Thoughts

Building this project has helped to cement alot of my learning from my AWS certifications. I have enough knowledge to feel confident I can build and help scale a platform whilst keeping costs down.

It's also helped improve my knowledge of Frontend development, particularly in React. I'd encourage anyone wanting to development their skills in cloud development to come up with their own project and get building. I'm incredibly proud of EV Buddy and will continue to tinker with it as I please.

You can try out EV Buddy at https://evbuddy.io/

Jetpack Compose for Maps

Darryl Bayliss — Wed, 09 Aug 2023 19:00:48 +0000

This post is based off a talk given at Google I/O Extended for Google Developers Group London in July 2023. The slides for the talk are available here

It’s hard to imagine Jetpack Compose 1.0 was released in July 2021. Fast forward two years and with 24% of the top 1000 apps on Google Play adopting Compose it’s easy to understand why.

Amongst all the excitement, one corner of Modern Android Development I feel recieves little attention is Google Maps. It’s been a while since I used the SDK, so was pleasantly surprised to see Google Maps was catching up with the times and released their own Compose library.

This will be welcome news to companies & engineers working in the mapping space. Mobile mapping is a $35.5B industry, with predictions of it raising to $87.7B by 2028. An Compound Annual Growth Rate (CAGR) of 19.83%. Source

Why is this important? A bigger market means more opportunities for companies to derive revenue from applications of mobile mapping. These range from the usual use cases, food, grocery delivery and ride hailing services. If you dig deep however, there are applications that aren’t immediately obvious. Below are the examples I could find after a brief search.

Mobile maps are great for Smart Cities, helping to manage the heartbeat of a city and visualising data in a way to better understand and react to its challenges. Useful for city planners, emergency response organisations or everyday residents.

Resource management also benefits from mapping solutions. Ranging from agriculture to fishing, mining to forestry, maps provide those in this line of work a perspective to make the right decisions to harvest materials in a sustainable way.

Transportation relies heavily on mapping technology. Not just consumer apps like Google Maps or Uber, but business level functions like understanding what a businesses fleet of vehicles are located. Transportation agencies also use maps to manage traffic and help make decisions on where to direct traffic to ease the flow.

Finally, with climate change and the weather being increasingly unpredictable, maps allows meteological agencies, emergency response units, and wildlife conservationists to understand how our world is changing and what we can do to take positive steps to reduce this.

Sources: Mordor Intelligence, GMInsights, Allied Market Research, EMR Research, Google Earth Outreach, Research & Markets

With the world providing more and more data, it’s a good time to learn how to put that data on a map. Let’s do that and get back to the code.

Using Google Maps for Compose

Google Maps for Compose relies on the following dependencies:

dependencies {
  implementation "com.google.maps.android:maps-compose:2.11.4"
  implementation "com.google.android.gms:play-services-maps:18.1.0"

  // Optional Util Library
  implementation "com.google.maps.android:maps-compose-utils:2.11.4"
  implementation 'com.google.maps.android:maps-compose-widgets:2.11.4'

  // Optional Accompanist permissions to request permissions in compose
  implementation "com.google.accompanist:accompanist-permissions:0.31.5-beta"
}

Google Maps for Compose is built on top of the Google Maps SDK, so you need to import the Compose library and the maps SDK. You won’t need to use most objects in the Google Maps SDK, as the compose library wraps most of these in Composables.

The utils and widgets libraries are an optional dependency. The utils library provides the ability to cluster markers on the maps, whilst widgets provides additional UI components. You’ll see these in use later.

For this post, I’ve included the request permissions library from Accompanist to demonstrate how to request location permissions, an often used permission with maps. Accompanist is an experimental library for Google to try out and gather feedback for features not yet part of Jetpack Compose.

Finally, you need to go to the Google Developer Console, sign up for a Google Maps SDK API key, and add it to your project. There’s a guide on the Google Maps Developer Docs on how to do this.

Security Tip: In the Google Developer Console lock down your API key so it only works with your application. This is avoids any unauthorised use.

Showing a Map

Showing a map is as simple as below:

 setContent {
    val hydePark = LatLng(51.508610, -0.163611)
    val cameraPositionState = rememberCameraPositionState {
        position = CameraPosition.fromLatLngZoom(hydePark, 10f)
    }

    GoogleMap(
        modifier = Modifier.fillMaxSize(),
        cameraPositionState = cameraPositionState) {
            Marker(
                state = MarkerState(position = hydePark),
                title = "Hyde Park",
                snippet = "Marker in Hyde Park"
            )
        }
 }

Create a LatLng object with the position of an area, and use it in conjunction with rememberCameraPositionState to set the initial position of the camera. This method remembers the position of the map as you move about using your hands or programmatically. Without this method Compose would recalulate the map back to its initial position on every state change.

Next, create a GoogleMap compose and pass in a modifier of your choice and the camera state. GoogleMap also provides a Slot API to pass in additional composables, these composables are what you want to draw on the map.

Add a Marker composable, then add a MarkerState containing the position of the marker inside. Finally, add a title and description of the marker.

Running this gives a nice aerial view of West London with a marker in Hyde Park.

Customising the Marker Window

You can customise the window of the marker by using a MarkerInfoWindowContent Composable. This also has a slot based API, meaning you can pass in your own composables to render your custom UI in the window.

setContent {
    val hydePark = LatLng(51.508610, -0.163611)
    val cameraPositionState = rememberCameraPositionState {
        position = CameraPosition.fromLatLngZoom(hydePark, 10f)
    }

    GoogleMap(
        modifier = Modifier.fillMaxSize(),
        cameraPositionState = cameraPositionState) {
            MarkerInfoWindowContent(
                state = MarkerState(position = hydePark),
                title = "Hyde Park",
                snippet = "Marker in Hyde Park"
            ) { marker ->
                Column(horizontalAlignment = Alignment.CenterHorizontally) {
                    Text(
                        modifier = Modifier.padding(top = 6.dp),
                        text = marker.title ?: "",
                        fontWeight = FontWeight.Bold
                    )
                    Text("Hyde Park is a Grade I-listed parked in Westminster")
                    Image(
                        modifier = Modifier
                            .padding(top = 6.dp)
                            .border(
                                BorderStroke(3.dp, color = Color.Gray),
                                shape = RectangleShape
                            ),
                        painter = painterResource(id = R.drawable.hyde_park),
                        contentDescription = "A picture of hyde park"
                    )
                }
            }
        }
}

Running this shows the custom window above the marker when you tap on it.

Showing multiple markers

Showing multiple markers is as simple as adding as many as you need. Let’s add markers for a few different parks in West London.

 setContent {
    val hydePark = LatLng(51.508610, -0.163611)
    val regentsPark = LatLng(51.531143, -0.159893)
    val primroseHill = LatLng(51.539556, -0.16076088)
    val cameraPositionState = rememberCameraPositionState {
        position = CameraPosition.fromLatLngZoom(hydePark, 10f)
    }

    GoogleMap(
        modifier = Modifier.fillMaxSize(),
        cameraPositionState = cameraPositionState) {
            // Marker 1
            Marker(
                state = MarkerState(position = hydePark),
                title = "Hyde Park",
                snippet = "Marker in Hyde Park"
            )
            // Marker 2
            Marker(
                state = MarkerState(position = regentsPark),
                title = "Regents Park",
                snippet = "Marker in Regents Park"
            )
            // Marker 3
            Marker(
                state = MarkerState(position = primroseHill),
                title = "Primrose Hill",
                snippet = "Marker in Primrose Hill"
            )
        }
 }

Run the code and you will see your markers appear on the map.

Clustering Markers

A map can get busy within a short amount of time. If you’re trying to show 300 markers, it’s going to be visually hard for a user to understand what is going on. Google Maps and your device won’t thank you ether, since it will have to render every single marker, impacting performance and battery life.

The solution to this is Clustering, a technique grouping markers close to each other into a single marker. This Clustering happens on a zoom level basis. As you zoom the map out the markers will group together into a cluster, as you zoom in the cluster will split into individual markers.

Google Maps for Compose provides this out of the box via a Clustering composable. There’s no need to write complex sorting or filtering for the clustering to occur.

 setContent {
    val hydePark = LatLng(51.508610, -0.163611)
    val regentsPark = LatLng(51.531143, -0.159893)
    val primroseHill = LatLng(51.539556, -0.16076088)

    val crystalPalacePark = LatLng(51.42153, -0.05749)
    val greenwichPark = LatLng(51.476688, 0.000130)
    val lloydPark = LatLng(51.364188, -0.080703)
    val cameraPositionState = rememberCameraPositionState {
        position = CameraPosition.fromLatLngZoom(hydePark, 10f)
    }

    GoogleMap(
        modifier = Modifier.fillMaxSize(),
        cameraPositionState = cameraPositionState) {

            val parkMarkers = remember {
                mutableStateListOf(
                    ParkItem(hydepark, "Hyde Park", "Marker in hyde Park"),
                    ParkItem(regentspark, "Regents Park", "Marker in Regents Park"),
                    ParkItem(primroseHill, "Primrose Hill", "Marker in Primrose Hill"),
                    ParkItem(crystalPalacePark, "Crystal Palace", "Marker in Crystal Palace"),
                    ParkItem(greenwichPark, "Greenwich Park", "Marker in Greenwich Park"),
                    ParkItem(lloydPark, "Lloyd park", "Marker in Lloyd Park"),
                )
            }

            Clustering(items = parkMarkers,
            onClusterClick = {
                // Handle when the cluster is tapped
            }, onClusterItemClick = { marker ->
                // Handle when a marker in the cluster is tapped
            })
        }
}

data class ParkItem(
    val itemPosition: LatLng,
    val itemTitle: String,
    val itemSnippet: String) : ClusterItem {
        override fun getPosition(): LatLng =
            itemPosition

        override fun getTitle(): String =
            itemTitle

        override fun getSnippet(): String =
            itemSnippet
}

Notice the ParkItem data class added. We need this because items passed into a Clustering composable have to conform to the ClusterItem interface. The interface provides the Cluster with the position, title and snippet for each marker.

Zoom in and out, and you’ll see the clustering in action.

Getting Location Permission

Maps and user position often go hand in hand, so it makes sense for some mapping apps to ask permission for the users location.

Treat a users permission with respect if you do this, location permission is one of the most sensitive permissions to acquire from a user. Make sure you inform the user of why you need this permission and actively show the benefits of granting it. Bonus points if your app partially functions without the need for permission at all.

Google provide some great guides on how to handle users location, as well as a separate guide for accessing location data in the background.

So you’ve done your due diligence and decided you do need the users permission to access location. With the permissions library in Accompanist you do it like so:

// Don't forget to add the permissions to AndroidManifest.xml
val allLocationPermissionState = rememberMultiplePermissionsState(
    listOf(android.Manifest.permission.ACCESS_COARSE_LOCATION,
           android.Manifest.permission.ACCESS_FINE_LOCATION)
)

// Check if we have location permissions
if (!allLocationPermissionsState.allPermissionsGranted) {
    // Show a component to request permission from the user
    Column(
        horizontalAlignment = Alignment.CenterHorizontally,
        verticalArrangement = Arrangement.Center,
        modifier = Modifier
        .padding(horizontal = 36.dp)
        .clip(RoundedCornerShape(16.dp))
        .background(Color.white)
    ) {
        Text(
            modifier = Modifier.padding(top = 6.dp),
            textAlign = TextAlign.Center,
            text = "This app functions 150% times better with percise location enabled"
        )
        Button(modifier = Modifier.padding(top = 12.dp), onClick = {
            allLocationPermissionsState.launchMultiplePermissionsRequest()
        }) {
            Text(text = "Grant Permission")
        }
    }
}

Via accompanist we’re checking to see if the app has access to the ACCESS_FINE_LOCATION permission, or a high level of GPS accuracy in english. It’s important to include the requested permissions in the Android manifest, as you won’t be able to request the permissions otherwise. The Android system and Google Play store also use the manifest to understand how your app works and inform users.

If permission isn’t granted, a small dialog composable is shown explaining the need for the permission and a button to launch the permission request via the system.

Animating the Map

Whilst most map apps require a user to move the map via touch, Google Maps for Compose provides APIs to move the map programmatically. This can be useful if you want to navigate to an specific area in response to an event.

In this example, we’ll gently navigate the app through our collection of markers.

Box(contentAlignment = Alignment.Center) {
    GoogleMap(
        modifier = Modifier.fillMaxSize(),
        cameraPositionState = cameraPositionState
    ) {
        Clustering(items = parkMarkers,
            onClusterClick = {
                // Handle when the click is tapped
                false
            }, onClusterItemClick = { marker ->
                // Handle when the marker is tapped
            })

        LaunchedEffect(key1 = "Animation") {
            for (marker in parkMarkers) {
                cameraPositionState.animate(
                    CameraUpdateFactory.newLatLngZoom(
                        marker.itemPosition, // LatLng
                        16.0f), // Zoom level
                      2000 // Animation duration in millis
                    ),
                    delay(4000L) // Delay in millis
            }
        }
    }
}

The key part here is the code inside the LaunchedEffect. For each marker the app sets up a cameraPositionState.animate() call to navigate to the marker. The camera receives this information via an update to the camera, created using CameraUpdateFactory.newLatLngZoom().

This method takes a LatLng, a float indicating the zoom level of the map and a long to set the duration of the animation.

Finally, to space the animations we use delay() to add a 4 second pause between each animation.

Showing Street View

It’s not just an aerial map Google Maps for Compose helps you with. You can also give apps access to Street View, showing a 360 view of a location. You do this by using the StreetView composable:

var selectedMarker: ParkItem? by remember { mutableStateOf(null) }

if (selectedMarker != null) {
    StreetView(Modifier.fillMaxSize(), streetViewPanoramaOptionsFactory = {
        StreetViewPanoramaOptions().position(selectedMarker!!.position)
    })
} else {
    Box(contentAlignment = Alignment.Center) {
        GoogleMap(
            modifier = Modifier.fillMaxSize(),
            cameraPositionState = cameraPositionState
        ) {
            Clustering(items = parkMarkers,
            onClusterClick = {
                // Handle when the cluster is clicked
                false
            }, onClusterItemClick = { marker ->
                // Handle when a marker in the cluster is clicked
                selectedMarker = marker
                false
            })
        }
    }
}

In this example, we’re setting the selectedMarker variable whenever a marker is tapped. If a marker is selected we show Street View, passing in the position of the marker.

Drawing Shapes

You may want to draw your own shapes and annotations onto the map. Google Maps for Compose provides a number of composables to do this, in this post we’re going to use the Circle composable.

A circle is a good shape to use if your app uses Geofences to react to changes from a users location. A circle can represent the area a geofence is active within.

 Box(contentAlignment = Alignment.Center) {
        GoogleMap(
            modifier = Modifier.fillMaxSize(),
            cameraPositionState = cameraPositionState
        ) {
            Clustering(items = parkMarkers,
            onClusterClick = {
                // Handle when the cluster is clicked
                false
            }, onClusterItemClick = { marker ->
                // Handle when a marker in the cluster is clicked
                selectedMarker = marker
                false
            })
        }
    }

    parkMarkers.forEach {
        Circle(
            center = it.position,
            radius = 120.0,
            fillColor = Color.Green,
            strokeColor = Color.Green
        )
    }

Here, we setup a circle for each of our markers. Creating a circle involves passing it a position and the size of the radius for the circle. We also use two optional parameters to set the color of the border and fill color for the circle.

Showing a ScaleBar

A good map often comes with legends and diagrams showing what a measure of space on the map is equivalent to in distance. This gives you an idea of the spaces involved in the map, as not every map may use the same form of measurements.

For digital maps that can zoom in and out, this adds a particular layer of complexity as the distances represented can change dynamically. Fortunately, Google Maps for Compose has you covered.

Using the Widgets library, you gain access to the DisappearingScaleBar and ScaleBar composables. These are UI components that sit at the top of the map, providing users with a measure of distance that changes depending on the zoom level.

 Box(contentAlignment = Alignment.Center) {
        GoogleMap(
            modifier = Modifier.fillMaxSize(),
            cameraPositionState = cameraPositionState
        ) {
            // You can also use ScaleBar
            DisappearingScaleBar(
                modifier = Modifier
                .padding(top = 5.dp, end = 15.dp)
                .align(Alignment.TopStart),
                cameraPositionState = cameraPositionState
            )

            Clustering(items = parkMarkers,
            onClusterClick = {
                // Handle when the cluster is clicked
                false
            }, onClusterItemClick = { marker ->
                // Handle when a marker in the cluster is clicked
                selectedMarker = marker
                false
            })
        }
    }

    parkMarkers.forEach {
        Circle(
            center = it.position,
            radius = 120.0,
            fillColor = Color.Green,
            strokeColor = Color.Green
        )
    }

After adding the composable, you get a ScaleBar that changes depending on the zoom level at the top of the map.

Help and Support

Google Maps for Compose is a great way to work with Google Maps and there’s plenty more to learn. Here are a few places I recommend if you need help:

Google Maps for Compose Repo: The repo containing the source code for the library. Contains code samples on how to use the library and also where you can submit your bug reports and contributions

Google Maps for Android Website: The place to go to learn the concepts behind Google Maps for Android. These are high level and don’t use the Compose library, but are nevertheless important to know as these are used in the background.

Google Maps Platform Discord The official Discord server for Google Maps. Here you can find people discussing Google Maps for multiple platforms, asking and offering help, and showcasing their own work.

DEV Community: Darryl Bayliss

Running a RAG powered language model on Android using MediaPipe

What is RAG?

Setting up your Android App

Creating Embeddings

Passing Prompts to your RAG Powered Model

Fine tuning your RAG Powered Language Model

Where To Go Next?

Multiplatform Development for Apple Devices

Adding A New Platform

Conditional Compilation Blocks

Dependency Injection via Build Phases

Adding App Bundle Resources

Where To Go Next?

Getting Setup With Xcode Cloud

Setting Up Xcode Cloud

Creating a Workflow

Start Conditions

Workflow Actions

Post Actions

Saving Your Workflow

Running Shell Scripts In Xcode Cloud

Viewing Your Workflow In Xcode And App Store Connect

Costs

Further Reading

Finally

Which Map Transformation Should I Use?

The Map Transformation

Map Transformations and Null Values

Acquiring an Index with Map Transformations

Other Transformations for Map Types

Passing Map Transformations to a Destination

More Resources

Playing Simon Says with Gemma-2b and MediaPipe

The App

MediaPipe

Putting MediaPipe to Work

Creating an Android Emulator with Increased RAM

Gemma 2b & Gemma 7b

gemma-2b-it-cpu-int4

gemma-2b-it-cpu-int8

Impressions and Thoughts

The Future of on Device LLMs

Useful Resources

Footnotes

Interacting with Apple Maps Through URL Parameters

The Apple Maps URL Scheme

Adding a Destination

Choosing the Transport Type

Selecting the Map Type

Further Reading

Building EV Buddy

EV Buddy

The Technical Stack

Frontend

Backend

Cloud Infrastructure

Choosing a Web Framework

Deciding on a Mapping Provider

Node.js, Serverless and AWS Lambda

Route 53

AWS Certificate Manager

S3 & Cloudfront

Costs

Improvements

Closing Thoughts

Jetpack Compose for Maps