DEV Community: HeadSpin

Introducing HeadSpin Local

HeadSpin — Mon, 30 Nov 2020 20:05:00 +0000

In the past few years, and even in just the past few months, we’ve witnessed rapid digital transformation across all industries, as changing consumer expectations present new opportunities for experience innovation. With this comes the need of product teams for a seamless, local approach to application testing and development, as they find themselves facing faster time-to-market pressure and their users increasingly distributed across the globe.

Conventional tools tend to fall short of delivering the velocity and scale, as well as the global reach, required for success in today’s experience economy. That is why we’re proud to introduce HeadSpin Local, a new platform that allows you to test your mobile experiences around the world from an easy-to-use platform powered by HeadSpin’s 24/7 technology.

With HeadSpin Local, you can perform manual testing on real iOS and Android devices running on real carrier networks in 9 locations worldwide — all from the comfort of your home or local office. HeadSpin Local provides a remote control workbench with full support for gestures and orientation that allows you to upload and debug your apps, games, and other mobile experiences on our pool of real mobile devices without modifications to your app. Plus, you can collaborate with ease by saving and sharing screenshots and device logs with your teams.

HeadSpin Local was designed to make testing easy and accessible to anyone and offers the following features:

Manual Mobile Testing

Identify bugs and issues faster with live, interactive testing.

Real Global Devices

Get 24/7 access to real iOS and Android devices where your users are: Palo Alto, New York, Miami, London, Johannesburg, Jakarta, New Delhi, Mumbai, and Tokyo.

Remote Workbench

Run and control your tests from our easy-to-use remote control workbench.

Screenshots

Device logs

Try it for yourself

HeadSpin Local is now available. Whether you’re launching a new app, want to test a new feature, or need to do a spot check, HeadSpin Local has you covered. Seamlessly test around the globe on real devices and real networks by running your first HeadSpin Local test today! Both annual and monthly subscriptions are available, and for a limited time only, you can get 25% off an annual subscription.

What’s next

HeadSpin remains committed to providing QA/release engineers, developers, product managers, SREs, and support professionals with the tools to collaborate seamlessly to accelerate development cycles, optimize digital performance, and automatically surface insights — all of which are especially crucial in today’s digital landscape.

Soon, teams will have access to the above features plus more, as our team works to bring HeadSpin’s core automated testing and performance analysis capabilities to users across the globe. Get updates here.

The post Introducing HeadSpin Local appeared first on HeadSpin.

I've Joined HeadSpin as Director, Learning and Education Programs

HeadSpin — Mon, 04 May 2020 17:26:54 +0000

It was over two years ago now when I started Cloud Grey, my Appium and mobile testing-focused consulting firm. It's been an amazing few years, and I've learned a ton by working with all my various clients. One of my major efforts has involved increasing the amount and quality of Appium training and education. My weekly blog & newsletter Appium Pro has grown radically over this time period because, I think, it has been meeting a need.

The need for high-quality training

Increasingly, I've found that what my clients really need is solid training for their staff. For that reason, earlier this year I launched a series of public workshops designed to take attendees from basic knowledge of Appium to an advanced level. At the same time, the world of mobile testing has continued to expand. The types of testing we're responsible for is growing; performance, network, and visual testing are increasingly non-optional components of any test framework.

In my role as a consultant, I've been simultaneously inspired and frustrated. There is so much inspiration to be found in client work and training—so many ideas to make Appium better, for one. It can also be frustrating because so much of my time and energy is put into managing the consulting machinery. Every hour I spend doing expenses or reviewing legal documents is an hour that I can't spend on Appium, or even producing more training-related content. So many people have asked me to create online courses, but I simply don't have the time! I realized that if I wanted to adequately reach the testing world with the type of high-quality training materials I had in mind, I'd need to rely on the energy of a whole team, not just my own.

Making the best quality training ever--with the help of a team!
Observing all these things, and trying to pay attention to them, I decided to join up with HeadSpin as Director, Learning and Education Programs, to find a better way forward for me, for mobile testing, and for training in this space. HeadSpin is a company I've worked with for over two years. I've been an advisor to the company on their Appium support and other initiatives. What I realized over this time is that HeadSpin believes wholeheartedly in Appium's vision, and wants to push it forward with an energy I haven't found anywhere else. HeadSpin is also exploring new products and technologies in the mobile testing space at a really exciting pace.

In addition, HeadSpin believes strongly in high-quality Appium training. So, there was a natural opportunity for us to talk about working together, with the mission of producing the best mobile testing courses available (both online and in-person). Specifically, we care about training that helps learners make practical, high-quality contributions to their companies' app testing. These conversations with HeadSpin began a long time ago, but now, given the huge need all around the world for good online training due to the requirement for many of us to stay home and help fight COVID-19, it's time to turn these ideas into reality! Stay tuned for more news from me on this front soon—I've got some exciting ideas about how to make this training both high-quality and widely accessible.

A taste of what's to come

In the meantime, I'm happy to announce that today we're launching a sort of pre-release version of what we're calling the HeadSpin University, where we've packaged up some of the webinars and articles I've developed in conjunction with HeadSpin over the years. Check it out here, and make sure to sign up for more news as it's made available. In the next few weeks, for example, I'm going to take some of what I've developed for the Appium Pro Workshops and record it as a set of training videos, making it available for free to the community. We all need a little help with everything that's going on in the world, and this is one way I want to give back.

Appium Pro is here to stay

With this change of employment and mission for me, I welcome the opportunity to have a simpler focus, which will also allow me to be more active with the Appium maintainers, pushing forward projects like Appium 2.0 or the creation of new drivers. As a consultant, I didn't have too much time to help manage the Appium project with the other maintainers, and I've lamented my inability to contribute as much as I used to.

I understand that this decision to join HeadSpin might raise some questions about the future of Appium Pro, or indeed of its stance in the industry. For my part, I'll continue to publish Appium Pro, and to write about topics in exactly the same balanced and neutral way I did as an independent consultant. HeadSpin will continue to be the lead sponsor, of course, and I'll continue to choose topics I care about and write what I want. It's been a wonderful resource for the community and I hope it continues to grow as a valuable repository of all things Appium. Indeed, even the trainings I develop under the HeadSpin banner will exist for the purpose of giving everyone the best quality training possible, regardless of the cloud service they ultimately want to use, if any.

I'm very excited about what lies ahead, and how this partnership with HeadSpin will enable me to bring more focus and more value overall to the Appium community. Please don't hesitate to let me know what you're hoping this new development will bring, too!

Jonathan Lipps

How to Integrate Audio APIs into your Appium Automation Scripts

HeadSpin — Wed, 29 Apr 2020 21:58:56 +0000

Have you ever had trouble testing the audio functionality of your mobile application? Capturing, injecting, and comparing audio has historically been a tricky area in mobile automation testing. Even trying to validate something as simple as checking if the correct audio plays during a prescribed action can be a daunting task. HeadSpin’s Audio APIs simplify these operations and provide developers with a convenient RESTful interface. This post walks through a high-level overview of each API as well as showcases how to integrate it into your Appium automation script.

What are these APIs? How do I use them?

In total, Headspin has seven different APIs regarding audio on mobile devices. This post will primarily focus on three very popular endpoints, which include uploading audio to our servers, capturing sound on a given mobile device in the HeadSpin cloud, and comparing two audio files to see if they are a match.

Uploading

The primary use of this endpoint is to allow the developer to upload a reference file that is used later to compare against a test audio file captured from the mobile device during your automation test. One thing to note is that the audio format for this endpoint is .wav. The response from this endpoint is a JSON object notifying if the upload was successful, and if it were, it would provide you with a unique audio id.

Example Code

def upload_reference_file(api_token):
api_endpoint = 'https://api-dev.headspin.io/v0/audio/upload'
data = open('reference_audio.wav', 'rb')
r = requests.post(api_endpoint,
headers={'Authorization': 'Bearer {}'.format(api_token)},
data=data)
response = json.loads(r.text)
return response['audio_id']

Capturing

You invoke this endpoint when you want to start capturing the audio output from a given device, and the API also allows you to end the session in a couple of different ways. When you make the initial capture call to the server, you can specify how long you would like it to run, or you can store the response, which includes a worker id and use it to poll for status updates and stop the capture as well.

After it has finished capturing the audio from the device, it is uploaded to a storage system that allows anyone in your organization access it. In the response from the capture endpoint, the audio id is passed back, which is essential to store for reference later.

Example Code

def capture_audio(device_address, duration, api_token):
api_endpoint = 'https://api-dev.headspin.io/v0/audio/capture/start'
data = {}
data['device_address'] = device_address
data['max_duration'] = duration
data = json.dumps(data)
r = requests.post(api_endpoint,
headers={'Authorization': 'Bearer {}'.format(api_token)},
data=data)
response = json.loads(r.text)
return response['audio_id']

Analysis

The analysis or match API allows you to pass in two audio files, a reference, and a test, and then determine if the reference audio is present in the test audio. We define the reference file to be the original audio source while the test audio is a more extended captured audio that contains the reference.

The use case here is to detect exact audio matches as well as locate the reference audio inside of the test audio. Additionally, it can also compare the audio quality of the test relative to the reference.

The response from this API includes multiple result parameters from the analysis. The key takeaways for me in this response is if it was a success first and foremost. Success does not indicate if the audios match but instead indicates that the algorithm was able to run correctly. Along with success, there are two objects in the form of parameters and results inside our response. Parameters described the thresholds for values used during the analysis, e.g. the sample rate. The most important behind a successful analysis is, of course, the results. The results object gives us the following stats:

Match: Full, Partial, No, or Error
Quality of Match
Start and End time of reference in terms of test

Example Code

def compare_audio(test_id, reference_id, api_token):
api_endpoint = 'https://api-dev.headspin.io/v0/audio/analysis/match'
data = {}
data['test_audio_id'] = test_id
data['ref_audio_id'] = reference_id
data = json.dumps(data)
r = requests.post(api_endpoint,
headers={'Authorization': 'Bearer {}'.format(api_token)},
data=data)
response = json.loads(r.text)
if response['success'] == True:
Return response['result']

How can we tie all of these together?

To showcase how we can integrate all of these endpoints into a single automation run, we are going to look at a customer use case. The goal of this customer was to verify that the correct automated response was given when a user of their service made a call with no balance on their sim card.

Given the proper reference audio, this is made relatively straightforward with the use of Headspins audio APIs. We executed this test in the following steps:

Upload a reference file to Headspin's storage system using the upload endpoint and store the audio id for later use.
Launch the Android/iOS device in Headspins mobile device cloud, which has audio enabled.
Navigate to the device's phone application using the Appium framework.
Enter a given number and place the call using a sim card provided by the customer.
Once the call connects, hit Headspins capture endpoint with a max duration of 20 seconds to record the automated response and store the audio id in the response for later use.
Send both the reference and test audio ids to Headspins analysis API to verify they are a match.

We can add this video as a visual walkthrough of the use case as well
https://www.loom.com/share/4ea534985a1a4f2187d6792566e52408

Writing an Appium test in Kotlin

HeadSpin — Mon, 27 Apr 2020 05:05:08 +0000

Kotlin is a modern programming language that focuses on conciseness, clarity, and code safety. Google officially adopted Kotlin by adding support into Android Studio in 2017 and since then has announced it as the preferred language for Android developers at Google I/O 2019. Perhaps your Android app team has incrementally added Kotlin code to your project or have elected to start new Android projects in Kotlin moving forward. Having the app code and UI automation code in the same language helps engineering and QA teams level set on the critical user journeys being tested. I’d like to share the basics of getting started with writing an Appium test in Kotlin. I’ll be using the Google News Android app as the target for this example and test on a real Google Pixel 3 XL from the HeadSpin platform.

Kotlin is fully compatible with Java so we can fortunately leverage the Appium Java client and JUnit libraries in addition to writing our code in the IntelliJ IDE. IntelliJ has a free community version called IntelliJ IDEA CE, which can be downloaded from their website or via your package manager.

Getting started

In IntelliJ IDE, navigate to File > New Project. Select Gradle and under Additional Libraries and Frameworks select Kotlin/JVM.

In the Project file view on the left, open the build.gradle file and add the following dependencies:

testCompile group: 'junit', name: 'junit', version: '4.13'

compile group: 'io.appium', name: 'java-client', version: '7.3.0'

We will add the Appium tests in the src/test/kotlin project directory.

Right click the kotlin subdirectory and select New > Kotlin File/Class. Create three new classes:

ProjectCapabilities
TestBase
GoogleNewsAndroidTest

We will structure the project in these three classes to make the organization clear. The ProjectCapabilities class will return the Desired Capabilities, TestBase will be a base class for our test for scaffolding the test setup and teardown, and lastly the GoogleNewsAndroidTest class will inherit from our TestBase, use the Desired Capabilities, and contain our test logic.

1. ProjectCapabilities Class

Let’s get started with the ProjectCapabilities class. This class will have a method to return the Desired Capabilities.

import org.openqa.selenium.remote.DesiredCapabilities

class ProjectCapabilities {

    companion object {

        fun AndroidBaseCapabilities(): DesiredCapabilities {

            val caps = DesiredCapabilities()

            caps.setCapability("autoAcceptAlerts", true)

            caps.setCapability("platformName", "Android")

            caps.setCapability("automationName", "UiAutomator2")

            caps.setCapability("deviceName", "8ABY0H6YG")

            caps.setCapability("udid", "8ABY0H6YG")

            caps.setCapability("appPackage", "com.google.android.apps.magazines")

            caps.setCapability("appActivity", "com.google.apps.dots.android.app.activity.CurrentsStartActivity")

            return caps

        }

    }

}

Functions and Methods in Kotlin are declared with the “fun” keyword. You can also indicate the function return type with a colon followed by the type. We can make this a static class method by declaring it in a “companion object.” You can create read-only variables with “val” and variables that can be reassigned with “var”.

In this case I am targeting a real Pixel 3 XL device in the HeadSpin platform. From the HeadSpin Web UI you can get the minimum Desired Capabilities for the device by clicking on the vertical dots menu and selecting Automation Configuration.

For the appPackage and appActivity I am specifying the associated values for the Google News Android app.

2. TestBase Class

Next, let’s work on our TestBase class. By default classes in Kotlin are implicitly final and can’t be inherited. To make a class inheritable, we’ll mark it with the “open” keyword. In this class we will create property variables for the: driver, capabilities, HeadSpin API Token, and WebDriver URL. We’ll provide visibility modifiers to our properties, in this case:

Protected: Only visible in the class or subclasses
Private: Only visible in the class

We’ll also mark our Desired Capabilities variable as “open” so it can be overridden from our GoogleNewsAndroidTest class which inherits from the TestBase class.

In addition you can indicate whether a variable can be null. The ability to declare this upfront is a great feature of the Kotlin language because it protects against unexpected NullPointerExceptions from our code. In this case we can indicate that the driver and caps can be null with “?” after the type and and set them to null. Kotlin also supports string interpolation by using the “$” character to interpolate a variable or “${}” to interpolate an expression. We’ll use this convenient language feature to pass our HeadSpin API Token to our WebDriver URL string.

Our TestBase will also have two JUnit annotations: @Before for our setup function and @After for our teardown to perform a driver quit. Our setup function will be straightforward and it will instantiate the driver object. In our teardown function we’ll check to see if the driver object is not null and if so quit our session. Otherwise we will throw an Exception. Kotlin provides a convenience operator for checking for a null condition with “?:”. This is called the Elvis Operator because if you turn your head sideways the question mark character looks like a swoosh of hair and the colon looks like eyes.

Do you see the resemblance now? Haha either way, the source code for the class can be found below and let’s carry on.

import io.appium.java_client.MobileElement

import io.appium.java_client.android.AndroidDriver

import org.junit.After

import org.junit.Before

import org.openqa.selenium.remote.DesiredCapabilities

import java.lang.Exception

import java.net.URL

open class TestBase {

    protected var driver: AndroidDriver<MobileElement>? = null

    protected open var caps: DesiredCapabilities? = null

    private val headSpinAPIToken: String = "your-api-token-here"

    private val webDriverURL: URL = URL("https://appium-dev.headspin.io/v0/$headSpinAPIToken/wd/hub")

@Before
fun setUp() {
    this.driver = AndroidDriver(webDriverURL, caps)
}

@After
fun tearDown() {
    this.driver?.quit() ?: throw Exception("Driver instance was unable to quit.")
}


}

3. GoogleNewsAndroidTest Class

Our last class, GoogleNewsAndroidTest will have our test logic. Here we have one test function annotated with the JUnit Test annotation. Our test will be simple and straightforward, we will tap on the Headlines tab button and then scroll down from the list of news articles. Since we will be performing driver actions we’ll need to first ensure the driver is not null. Kotlin provides a way to perform an operation only if the value is not null using the “?” (the safe call operator) together with “let.”

import io.appium.java_client.MobileBy

import org.junit.Test

import org.openqa.selenium.interactions.Interaction

import org.openqa.selenium.interactions.PointerInput

import org.openqa.selenium.interactions.Sequence

import org.openqa.selenium.remote.DesiredCapabilities

import org.openqa.selenium.support.ui.ExpectedConditions

import org.openqa.selenium.support.ui.WebDriverWait

import java.time.Duration

class GoogleNewsAndroidTest: TestBase() {

    override var caps: DesiredCapabilities? = ProjectCapabilities.AndroidBaseCapabilities()

    private val headlinesTabButton: String = "com.google.android.apps.magazines:id/tab_headlines"

@Test
fun headlinesScrollTest() {

    // Set an explicit wait of 10 seconds
    val wait = WebDriverWait(driver?.let { it }, 10)

    // Tap on the Headlines tab button
    wait.until(ExpectedConditions.presenceOfElementLocated(MobileBy.id(headlinesTabButton))).click()

    // Scroll Down
    val finger: PointerInput = PointerInput(PointerInput.Kind.TOUCH, "finger")
    val moveToStart: Interaction = finger.createPointerMove(Duration.ZERO, PointerInput.Origin.viewport(), 726, 2452)
    val pressDown: Interaction = finger.createPointerDown(PointerInput.MouseButton.LEFT.asArg());
    val moveToEnd: Interaction = finger.createPointerMove(Duration.ofMillis(1000), PointerInput.Origin.viewport(), 726, 660)
    val pressUp: Interaction = finger.createPointerUp(PointerInput.MouseButton.LEFT.asArg())

    val swipe = Sequence(finger, 0)
    swipe.addAction(moveToStart)
    swipe.addAction(pressDown)
    swipe.addAction(moveToEnd)
    swipe.addAction(pressUp)

    driver?.let { it.perform(arrayListOf(swipe)) }

}


}

Conclusion

Our work here is done! We can now execute our test and watch it in action.

Congratulations you now have a taste of what it’s like to write an Appium test in Kotlin! If this article has helped further your knowledge feel free to Like or Share it. You can download the full project source code here.

The post Writing an Appium test in Kotlin appeared first on HeadSpin.

How to Automate the Zoom Video Meetings App

HeadSpin — Thu, 23 Apr 2020 17:11:42 +0000

Last week I tried something new: Automation Happy Hour. In this live stream event, I downloaded the Zoom app for Android and tried to build a simple testsuite for it in about an hour. I think this type of real-world “case study” is really interesting. My expectation is that discussing automation of real apps “in the wild” would also be useful for Appium Pro readers, so periodically, I will take the results of these explorations and turn them into Appium Pro articles!

(NB: In the live stream, I chose to work in Python, but for this article I converted all the code examples to Java.)

Starting a session with Zoom

Zoom is incredibly popular. It seems like everyone is using it for their video meetings these days. How can we launch it in an automation context with Appium?

First, we need to decide whether to have Appium install the Zoom app for us, or to have the app on the phone and simply launch it when our session starts. I already had the Zoom app on my device, so I chose the latter route, but you could also find a downloadable Zoom APK, put it somewhere on your system, and reference it as the app capability.

Since I had the app already, I needed to know its package id and launching activity to get Appium to start it for me. Using the helpful ADB command adb dumpsys window windows, I determined Zoom’s info, and entered that into my set of capabilities:

caps.setCapability("platformName", "Android");

caps.setCapability("deviceName", "Android");

caps.setCapability("appPackage", "us.zoom.videomeetings");

caps.setCapability("appActivity", "com.zipow.videobox.LauncherActivity");

caps.setCapability("automationName", "UiAutomator2");

(The important bits above are the appPackage and appActivity capabilities). Using these caps, we can launch zoom in an automation context!

Joining a meeting

The goal is to do something useful with automation, so let’s see how it’s possible to join a Zoom meeting from a device controlled by Appium. Before we begin, we’ll obviously need a Zoom meeting to join! I started one on my computer, so that I could determine its meeting ID and password. These are the two pieces of information you’ll need that are unique to your case, and to work with my code examples, you’ll need to set them as environment variables, so that they can be read from our test class:

private static final String MEETING_ID = System.getenv("ZOOM_MEETING_ID");

private static final String MEETING_PW = System.getenv("ZOOM_MEETING_PW");

(I.e., you need to set the ZOOM_MEETING_ID and ZOOM_MEETING_PW environment vars to run the examples successfully, based on a meeting that you own or know about–and, please, no zoom-bombing!)

It turns out that Zoom is a highly automatable app! Its developers included resource IDs for all the useful elements. There were no content descriptions (accessibility IDs) that I could find, however, which makes me think the app may not be super accessible (or, it relies on the default accessibility strategies based on text labels).

To figure out how to find all the elements I needed to interact with, I used Appium Desktop to inspect the app (after filling out all the appropriate caps, and starting a session with the inspector). It turns out that the Zoom elements follow a pretty standard naming scheme. Here’s the part of our test class where we define the locators, for example:

private By JOIN_MEETING_BUTTON = By.id("btnJoinConf");

private By MEETING_ID_FIELD = By.id("edtConfNumber");

private By ACTUALLY_JOIN_MEETING_BUTTON = By.id("btnJoin");

private By PASSWORD_FIELD = By.id("edtPassword");

private By PASSWORD_OK_BUTTON = By.id("button1");

private By LEAVE_BTN = By.id("btnLeave");

You can see that, apart from PASSWORD_OK_BUTTON (which is referring to a system alert element), every UI element has a camelCase ID with an abbreviation of the element type at front. Pretty respectable, Zoom!

Using these locators, we can very straightforwardly walk through the join meeting flow, as the video below shows:

The test code that implements this is as follows (using the locators defined above):

@Test

public void testJoinMeeting() {

// navigate through the UI to join a meeting with correct meeting id and password

waitFor(JOIN_MEETING_BUTTON).click();

waitFor(MEETING_ID_FIELD).sendKeys(MEETING_ID);

driver.findElement(ACTUALLY_JOIN_MEETING_BUTTON).click();

waitFor(PASSWORD_FIELD).sendKeys(MEETING_PW);

driver.findElement(PASSWORD_OK_BUTTON).click();

// prove that we made it into the meeting by finding the 'leave' button

waitFor(LEAVE_BTN);

}

Here we are using a special waitFor helper which just reduces verbosity in the code:

private WebElement waitFor(By locator) {

return wait.until(ExpectedConditions.presenceOfElementLocated(locator));

}

(You might have observed that sometimes we use waitFor and sometimes we just use driver.findElement. Why is that? It wouldn’t hurt to use waitFor everywhere, but in some cases we know that an element will already be present, because we’ve already waited for another element on the same view. If that first element is not actually present immediately, then there’s no reason to wait however many seconds for it to become present. The test is doomed and we might as well fail quickly!)

So that’s it! You can as always check out the full code example as well. And if you’re interested in checking out some of the Automation Happy Hour live streams (or the video recordings of them), don’t forget to follow me on Twitter, where I announce these things as well as run polls on what apps I should automate in future episodes.

Oh, and don’t forget to let me know if you come up with any interesting (and non-nefarious) use cases for automating Zoom specifically!

The post How to Automate the Zoom Video Meetings App appeared first on HeadSpin.

Active Testing SMS with HeadSpin

HeadSpin — Thu, 12 Mar 2020 15:48:00 +0000

SMS testing is a challenge regardless of location, device, or network- due to inaccurate reporting and regional regulations it’s almost impossible to consistently and accurately generate reports. DLR, or Delivery Reporting, is a feature of SMS message termination that reports back if the SMS message has been delivered. Typically, this message is sourced directly from the handset.

Telecoms providers can report whether an SMS has been delivered, but with no regulations in reporting, much of the data is inaccurate. To make matters more complex, many of the reports that are generated can only report if the transaction was a success or failure.

In this post, we demonstrate how we can leverage the HeadSpin platform for SMS testing in three high-volume countries: India, China, and Brazil, and discuss how each can be improved.

What is “active testing”?

Active testing means your service sends an SMS to a phone or device that you control and then checks to see if the SMS was received and what aspects of the message changed in the process. However, the scope of active testing is larger than running unit or end-to-end testing. Test SMS messages can be sent ad hoc when setting up a new service, or for detecting delivery issues on a particular carrier or within a specific country. The classic approach to “active testing” is to call your colleague in India and tell him you’re sending a message. Understandably, this approach doesn’t scale.

What does active testing look for?

Active testing is the best tool for determining if a message sent via a provider will arrive at the destination looking like it is expected to.Active testing can identifymany of the following issues:

Character encoding issues
Consolidator substitution
Concatenation behavior
Content modification
Sender ID modification
Blocking or content filtering
- URLs
- Telephone numbers
- Email address
Actual delivery (remember, DLRs can lie)
Invalid/valid phone number support

With HeadSpin, you can identify all kinds of telecom pitfalls from the user’s perspective including :

Is your Alpha-Sender ID preserved? Shortcode?
Are your US long codes working in Europe? EMEA? APAC?
Are any of your numbers re-written?
Does your app signup really work in the middle of Africa?

HeadSpin’s POP, or Point of Presence, phones allow any support or engineering team to have a physical presence across the globe.

Examples of failures identified with HeadSpin and “active testing” include:

Identifying delivery failures in China.
Customer’s improper regex phone number configuration in African countries..
Identifying carrier rewrites in numerous countries.

Active SMS Testing in India, China, and Brazil

Let’s see how a Leading CPaaS provider’s general ( CPaaS SMS ) and turnkey ( CPaaS Turnkey ) solutions work out of the box to deliver in three high-volume countries – India, China, and Brazil. India has a heavily regulated telecom system with many rules around when and what you can send. China, on the other hand, is constrained by the great firewall which could, at any time, suspend or delay messages. China is also quite draconian if you send messages with illicit content. Doing so may very well result in the suspension or loss of your SMS sending privileges. Brazil has historically had issues with carriers and Unicode making it another fun country to experiment with.

We’ll use the CPaaS Turnkey API’s default configuration to send OTPs. For India, this means it should deliver to recipients via a registered Alpha Sender ID. For Programmable SMS, we’ll simply register a US long code with CPaaS provider to send directly to the HeadSpin POP handsets.

Once we’ve found a phone to test with, the very first step is to simply text your own phone as a reality check. Honestly, it’s pretty amazing to initiate an SMS from the other side of the world and watch it show up to your own local phone within seconds. This first step just demonstrates that everything is working.

Discovery Data Points

The intent of this testing is to see the real-world SMS experience a user has on their phone in each location. We will record each data point as below:

Mobile Number: Phone numbers associated with HeadSpin device
Device: What kind of device?
Location: Where is the device located?
Carrier: Carrier the device is using
Country to US: Are we able to send an SMS from India to the device in the US
CPaaS SMS to Country: Are we able to send an SMS to the device from the US via Programmable?
CPaas Turnkey to Country: Are we able to send an SMS to the device?
Expected Body: Did the carriers process the body correctly?
Message Body : “Hello from San Francisco?!”
Long Message Body: “ The lazy brown fox jumped over the fence. The lazy brown fox jumped over the fence. ……”
Unicode Body: ★ ☆ ☇ ☈ ☉ ☊ ☋ ☌ ☍ ☏ ☐ ☒ ☓ ☚ ☛ ☜ ☞ ☟ ☡ ☤
DNA = Did Not Arrive

India Testing

A Note 5 Pro device on HeadSpin’s public cloud is depicted below. This is one of 3 devices we’re using to test telephony delivery in India. Each device uses a different carrier network and supports both automated and manual QA testing via any testing framework.

Data from Testing in India

India Conclusion

Admittedly, this is a pretty small sample size but it gives you a general idea of what kind of experience a user can have with different carriers in each location.

The most obvious issue here is the non-delivery of SMSs to India via the CPaaS SMS API. This is expected behavior as delivery to India typically requires jumping through several technological and bureaucratic hoops.

The real winner in this small sample size is the CPaaS Turnkey. Obviously, they have nailed their implementation as everything was delivered in a timely manner. The interesting observation is that out of the six attempts, it seems only one was delivered via the Alpha Sender ID registered in India. This could be a side-effect of the intelligent retry logic or simply Indian carrier mischief.

China Testing

For testing in China, we focused on two major metro areas, Shanghai and Beijing. You’ll also note that each phone used a completely different carrier. DNA denotes, did not arrive.

Data from Testing in China

China Conclusion

Testing with HeadSpin in China resulted in the complete opposite experience from India. In this case, 33% of Programmable SMS failed to arrive, and NO Verify messages sent to China arrive. Like most Voice or SMS API calls to phones terminating in China, misconfiguration or mischief by the Chinese firewall is surely to blame.

Interestingly enough, the message sent from China to the US arrived quicker than any other test.

Brazil Testing

Testing in Brazil was limited to a single carrier network. Overall, Brazil demonstrated the best performance across devices. The carrier behavior did not vary between phones and provided the fastest experience across all three test countries.

Data from Testing in Brazil

| Mobile Number | +55XXXXX | +55XXXXX | +55XXXXX |
| Device | iPhone XS | iPhone 6s | iPhone 7 |
| Location | São Paulo | São Paulo | São Paulo |
| Carrier | Vivo | Vivo | Vivo |
| India to US | No lag | No lag | No lag |
| CPaaS SMS to Brazil | No lag | No lag | No lag |
| Expected Body? | Yes | Yes | Yes |
| Unicode Issues? | No | No | No |
| Delivered By? | 27199 | 27199 | 27199 |
| Lag | No | No | No |
| CPaaS Turnkey > Brazil | No lag | No lag | No lag |
| Expected Body? | Yes | Yes | Yes |
| Unicode Issues? | No | No | No |
| Delivered By? | 27199 | 27199 | 27199 |
| Lag | No | No | No |

Brazil Conclusion

Testing in Brazil seems to have provided the best results out of the three countries. At the time of testing, there were limited Brazilian carriers to test with but on a positive note, the experience was exactly the same across each device.

Why Active Testing Matters to You!

Working with telecom is a fickle beast. When talking with customers, I frequently compare sending an SMS to sending a UDP packet. With both of these technologies, you initiate a send and the data is off! But, you never know if it has gotten to its destination. Any number of issues could affect the delivery.

Queueing (is there a football game in Munich, a cricket game in Lahore?)
Infrastructure Issues
Political Suppression
Grey Routes

Another way to think of SMS telephony is to consider hiring a container ship to deliver goods to a port of call. You can hire a cheaper company that takes the long route around South America or the more expensive company which goes through the Panama Canal. It is this route that has a far higher likelihood of arriving on time at the port of call. Your cheaper company may end up at the bottom of the sea while rounding Cape Horn.

As a mobile developer, you’ll have a clear understanding of how your mobile experience will play when leveraging HeadSpin’s robust suite of testing and performance monitoring solutions. You can take immediate control of a phone anywhere in the world with our Global Device Cloud and test to your heart’s content. Looking for a more robust long term solution? Leverage your own private cloud for automated testing or setup hourly, daily, weekly testing and benchmark reporting to identify trends and issues around your KPIs.

HeadSpin can help you automate functional, performance, and load testing across applications, devices, and networks to optimize digital experiences enabled by mobile, web, IoT, and 5G. Want to give HeadSpin a try? We have a Point of Presence in over 100 countries, 190 locations consisting of 30,000 devices. Reach out via our Contact Us form and we will coordinate a discussion and demo.

The post Active Testing SMS with HeadSpin appeared first on HeadSpin.

“Personal Trainer” for your mobile apps

HeadSpin — Tue, 28 Jan 2020 22:44:22 +0000

I remember the early days of using an app to track my runs. I diligently tried out 4-5 apps and then finally settled on one that was the most convenient to measure my run, and clearly displayed the metrics I cared about. Battery consumption is a crucial part of this experience, given that no one wants to lug around a portable battery pack while they are on their run.

However, it’s a no-brainer that there will be some sort of a tax on the smartphone, especially if your user is combining multiple apps and hardware experiences. Most folks who use these live-tracking apps use them in conjunction with their favorite media streaming apps — usually listening to music or a podcast over their headphones (wire or wireless). What’s important for any app developer is to identify and measure how their apps actually take battery and other aspects into account when building a seamless user experience. For example, how can app developers test these on-the-move experiences before they launch new features? How can they continue to track performance post-launch? And how can they do these things without an SDK, please?!?

At HeadSpin, we are constantly thinking of tools and performance insights that developers need to test their apps and games, on real devices anywhere in the world. Innovating upon our on-premise solution, we built the first-of-its-kind portable solution that provides results that accurately depict real-world conditions, be it testing on real devices while driving, commuting between places, or walking/running around pedestrian areas or events.

Our sleek AnyWear Mobility Unit is right up there with the best ideas that come in small packages! These ready-to-carry backpacks were a hit at CES 2020 and we are excited to offer them to our customers.

Last week, we took these backpacks on a stroll, around our Palo Alto Office, to test the top Running Apps in the Health & Fitness category. We made two trips under 10 mins each, and covered the exact same route for all the apps. With the HeadSpin AnyWear Mobility Unit, we were able to measure and compare app performance in terms of Net CPU%, Energy Drain% and Memory Usage%.

How is this possible, you ask?

The HeadSpin technology leverages network traffic, client-side metrics, and videos of the test execution to pinpoint areas of poor user experience and performance bottlenecks. We also correlated performance with precise GPS coordinates. The following devices were used over real network conditions:

Samsung Galaxy S9, on Sprint LTE for Android OS 8
Samsung Galaxy S9, on AT&T for Android OS 9
Samsung Galaxy S8 Plus, on Verizon for Android OS 7
Samsung Galaxy S9, on Verizon for Android OS 9

What did we find?

We analyzed the current published versions of the top 8 Running apps in the Health & Fitness category on 4 Android phones in the US.

Upon reviewing performance sessions post-walk, we noticed that all apps more or less captured the total distance accurately — except for Under Armor’s Run With Map My Run.

At one point the Run With Map My Runapp showed a significant issue in terms of tracking the path midway for the exact route that was mapped accurately by other apps.Guess the Run With Map My Run app from these screenshots gathered from the apps.

When we analyzed the other crucial performance indicators, we noticed some interesting results:

In terms of Net CPU% the Nike Run Club app consumed the highest CPU usage at 39% while Run with Map My Run was lowest at 28%.

In terms of

Memory Usage%, all apps are above 50% except for Run with Map My Run, with Endomondo, Strava and the Nike Run Club apps above 60%.

In terms of Energy Drain%, Google Fit consumed the most energy at 3.93% for a <10 min walk around the block.

During testing, these apps were the only ones running on the devices, but your typical user is probably running a few apps at the same time when they are on the go, and these numbers are only going to be exacerbated in those instances.
User attrition is a real threat to digital business. In today’s crowded app ecosystem, users can easily move to other, competing apps, and if they are dissatisfied with an app’s mobile performance, they won’t continue to use it over the long run.

This is precisely why developers need a more sophisticated platform to test and monitor the quality of connected experiences. The HeadSpin AnyWear Mobility Unit lets developers test connected devices the way that users experience them — on the go, in real time.

And since performance is always relative, we also offer HeadSpin Benchmark Reports if you are curious to see how other apps perform in comparison with your crucial KPIs.
Are you interested in taking your app or game out for a run and testing how they perform? Reach out to our team here, and we’ll get in touch!

Next up! How does your favorite Music Apps perform on a busy US Highway 101? Stay tuned for our insights!

The post “Personal Trainer” for your mobile apps appeared first on HeadSpin.

Our Experience at CES 2020

HeadSpin — Tue, 21 Jan 2020 11:00:36 +0000

Earlier this month, we attended CES 2020 for our very first time. We had a booth set up with our partner Wipro in the Enterprise Solutions section at the Las Vegas Convention Center, where we provided demos of our Connected Intelligence Platform™ to driven developers looking to deliver high quality global user experiences in an increasingly complex digital world. We had our proprietary Proxy System and portable mobile device cloud set up as well and were able to showcase their versatility to great success.

Eravi demonstrating our proprietary Proxy System

CES 2020 was a great way to start off the new decade. It was an honor to be able to meet with other innovators and pioneers with ideas set to shape consumers and business lives and transform the technological landscape.

Eravi and I all ready for Day 2 of CES2020

When we explored the show floor to witness the latest tech in action, we were able to connect with a diverse array of other innovators in the business as well. One overarching trend was clear: technology that connects the digital experience to the real world has become a greater focus now more than ever. All that we experience now is connected.

Here’s what our key takeaways were from CES 2020 and what we noticed were the most noteworthy trends we believe may define the industry this year:

Smart Home Technology is getting even smarter

Photo by CES

Smart Home is getting even smarter. We noticed a great focus on smart device and user experience integration this year. The major theme was the Smart City. Many of the biggest players had an offering for both the Smart Home and its integration into the Smart City. Smart home technology that connects appliances such as dishwashers, refrigerators, and assistants has been gaining quite a lot of attention in the past and were especially prominent this year. Among the many offerings: a voice-activated smart faucet.

Foldable display technology is taking flight

Photo by Marco Verch under CC 2.0

From smartphones to even speakers, foldable display technology seemed to be omnipresent at this year’s conference. Though its still in its early stages, this may be the year we see the technology start to resonate with and appeal to more consumers.

Let’s move on to the solutions

Photo by CES

AR and VR devices were a recurring theme at CES 2020. There was less of a focus on the holographic technology itself and more AR/VR solutions being presented to attendees this year.

Connected Experiences are for cars too

Photo by CES

As we had expected, another major theme was the autonomous car. There was a focus on smart technology in use for cars as well; Smart speakers and other IoT devices finally made their way into the vehicle this year. Companies went beyond the standard self-driving technology popular today and in previous years to debut features that make passenger experiences more connected and more personalized.

Photo by CES

Needless to say, CES 2020 was an absolute success.

Our team made a strong impact with our HeadSpin Connected Intelligence Platform™ and our proprietary Proxy System, demonstrating how both have provided our partners and developers with crucial web, mobile, IoT, and 5G solutions to optimize connected experiences that improve their consumer’s digital experience. Our platform was a big draw for attendees, from developers to Smart Home and City innovators, looking to bring the best digital experience to their users in 2020 and for years to come.

We are incredibly grateful to have been a part of CES 2020 and excited for our team’s upcoming ventures. We got a lot planned in 2020, including exhibiting at many other events. Stay connected with us on our Linkedin or Twitter for updates on where we’re heading to next.

We also recently conducted a test run with the AnyWear Mobility Units that we showcased at CES to measure and compare the performances of 8 of the top running apps.

Missed us at CES 2020 and want to learn more about our Connected Intelligence Platform? Learn more about it here or check out how we’ve helped to drive digital business success for some of today’s top companies.

The post Our Experience at CES 2020 appeared first on HeadSpin.

Dialing in faster app performance

HeadSpin — Mon, 25 Nov 2019 23:59:47 +0000

Users hate hourglasses, spinning wheels and other reminders that their precious time is wasting while apps and networks slowly grind away. Research is clear: waits of more than 2-5 seconds lead quickly to abandons and curses. Good developers understand the need to optimize mobile performance and user experience, both before and after production.

Fast apps and page loads are especially crucial in hyper-competitive industries like telecom. Janky, cranky mobile experiences aren’t exactly the best look for a leading-edge communications service provider (CSP) battling for new consumer and business subscribers.

So this fall, when a major multinational telco asked HeadSpin to run mobile performance testing in its key (and highly competitive) New Zealand and Australia markets, we jumped at the chance to help. We knew we could improve understanding of their network, application performance and user experience. We also wondered if reported billing troubles on their monolithic core system was also causing problems for people interacting with the company via its mobile app.

What we found is instructive in helping companies of all types spot and fix issues contributing to sluggish mobile experience. We’re sharing here as a part of our occasional posts showing how data-driven, pinpointed adjustments can yield big improvements in both customer experience as well as in technology and business performance.

ABOUT THE SESSION

HeadSpin conducted our tests on Monday, Oct. 21. As always, the goal was to help identify the root cause of high-priority performance issues, from the client-side all the way to server-side.

Our team generated a performance session, which was then analyzed by our AI-based issue-detection engine. We used an iOS 12.4.1 device located in Auckland, New Zealand on the company’s mobile network (Tests can be run on 32,000+ different devices across 160+ locations in 110+ countries). The scenario: A mobile customer is looking to check his balances, increase his data plan, and sign up for roaming for an upcoming trip to Japan.

We’ll do a quick walk through the results below. offers high-level session-wide metrics, domain metrics, burst metrics and host metrics.

Here's a snapshot of the testing setup:

A final note: We didn’t do (and never do) any rooting or jailbreaking to run the test. Both give you elevated system privileges on a device, and that’s something an ordinary user would never get or do. So no jailbreak for you.

THE EXPERIENCE

A hint that the app test might not break speed records comes early in the session. There’s an obvious lag in the form at :26 seconds, when our test customer enters a password. That’s followed by another noticeable experienced delay, when it takes a few seconds to log in. In the meantime, we’re left watching an animated white “working on it ” icon wiggle on a brightly`` colored background. It’s cute, but quickly irksome. As feared, the biggest delay of the session comes at 1:48, when our user tries to update his roaming services.

In the end, overall wait times for our app end-user exceeded 6.5 seconds. (For perspective, that’s enough time for a light beam to travel 1,210,833 miles, the equivalent of 48.75 trips around the equator). What on Earth was going on?

Here’s what all the tests looked like in context. Obviously, there are too many data and performance insights to fully explore here. So let’s focus on three key areas:

ISSUE 1 SLOW SERVERS

Server performance is a key foundation of mobile app performance. The most elegantly crafted app can be slowed by sluggish back-end performance. Research shows a straight line between wait times and user abandonment. Google/SOASTA found probability of bounce jumps to 123% with a 1-10 second delay in mobile page loads. Akamai calculatesthat a 2x increase in mobile load time cuts conversions by 50%. Too slow and users go.

That’s why it was concerning that our test run uncovered serious slow server issues. Delays in this foundational function were, unfortunately, a big contributor to overall delays experienced by our test app user. For instance, the app waited 4.23 seconds for the http POST request and 2.11 seconds for a GET request. Not good on either count.

As a rule, any wait time longer than 300ms to complete a handshake with the host is worth investigating. In one case, the app waited for as long as 394 ms. We confirmed the physical location of the laggard destination as Sydney, Australia. But it got worse: In another instance, we timed the wait at 982 ms. That server was in located in Utah. The southwest United States is a long way from New Zealand (more than 7,000 miles/11,300 km.), so that’s a likely culprit.

Remember that slow load we mentioned a minute ago? Here’s what it looks like to the user and the HeadSpin test system:

See the peak, two-thirds to the right? That’s your sloooow wait for the billing system described above.

This alternate view shows the slow server problem’s relative seriousness compared to other, cooler-colored issues.

To get a more granular idea of what’s impacting performance, take a look at this drill-down of analytics and ad services on the site. Impacts range from a relatively small 80ms. for Google to 543 ms. for a1/adform.net.

The Fix

Speeding up slow servers starts with a clear understanding of the resources that hosts are serving to your app. Are you expecting these hosts to be slow? You’ll want to determine if they’re performing a lot of server-side work before sending replies. If the hosts are part of a content delivery network, make sure they’re not serving resources from the wrong edge. And confirm that the physical location of the request's destination IP is where you think it is.

ISSUE 2 LOW PAGE CONTENT

Perhaps the only thing worse than a web page with no content is a page with low content. Issues here are closely related to the sever slowness described above. It’s why for more than a decade, Facebook has institutionalized best practices designed to do just that.

During our testing we found the end-user consistently experiencing little content on screen for more than one second at a time.

The associated delays contributed to a total of more than 5.5 seconds of wait time. With long network requests and network saturation as possible contributing factors, wait times reached 6.56 seconds in one instance as shown below:

The fix

Looking at the orange impact region in the Waterfall view lets HeadSpin correlate root cause factors, such as long network requests, network saturation, or usage in the system timeseries. Any abnormal metric in this region can be causing the issue.

In this case, it would be helpful to examine the keep-alive timeout. Proper setting ensures connections are re-used (as opposed to re-created) to avoid producing a sub-optimal user experience. So we did…

And indeed, testing found the app creating new TCP connections to the impacted host [www.the telco .co.nz:443], Since keep-alive connections is enabled, it’s wise to check that the timeout is not set too low, which could cause client connections to terminate prematurely, resulting in unnecessary TCP/TLS handshakes. Resetting the server ensures the TCP connection is not dropped.

ISSUE 3: SLOW ANIMATION LOAD

Animation on a mobile app can be tricky. Done well, it’s a delightful distraction during loads, judiciously recommended by UX gurus like Nielsen/Norman Group and Luke Wroblewski.

In our test, one of the (potentially) coolest parts of the test session (a customer increasing monthly data allocation) ended up, well, not so cool. Onscreen, the user was served with two spinning slot machine windows. “Grab the Prepay Deal of the Day!” The windows were supposed to stop, presumably revealing a great offer. Unfortunately, animation load and confirm time was slow (1.36 seconds on screen– long enough for a visitor to wonder if their luck had run out.

The fix

Problems with graphics loading often mean the app is waiting for a network resource. Looking around the Waterfall’s orange region reveals potential problems such as unusual, too much, or missing network traffic. One fix is to have app and server can improve the way the resource is loaded. Another way to speed up animation would be to rework the user design. Elements that load faster can be moved to the top, and slower elements relocated below the fold.

BOTTOM LINE

Slow loads are fast roads to losing customers. The advent of 5G will only further shrink the time that users will wait for both new and existing applications to load and run.

For the first time in 2019, consumers will spend more time on mobile phones than watching TV.

The average adult will spend 3 hours and 43 minutes a day on mobile devices, according to new research by eMarketer. Of that, 2:57 is spent on apps, the company says. Don’t miss out.

Performance testing offers a great, effective way to find and fix problem areas in production apps. The logic is simple: To fix performance, you need to measure performance. To measure performance, you need to see performance.

With our Digital Experience Platform, this telco gained valuable insights into their performance issues and steps to improve the end user’s experience in a highly competitive market and region. (An interesting postscript: A few days after our first test, came under criticism in the U.K for reported errors in roaming billing and unwanted customer disconnections.)

HeadSpin offers an enterprise-grade approach for global mobile experience. Our all-in-one platform for remote testing (with real SIM cards across 160+ locations), network testing, a complementary “mobile performance management”, and AI-driven approach simplifies pre-and-post launch improvements in real-world user environments.

Want to launch to launch flawless products faster? Identify and fix bottlenecks? Schedule a demo to learn more about how HeadSpin can help improve your customer's digital experience.

Tips for Shopping Apps this Holiday Season!

HeadSpin — Wed, 06 Nov 2019 06:28:14 +0000

Gone are the days when the majority of your customers stood outside the store eagerly waiting for the doors to open, only to rush in and grab all they need (and don’t need) during the holiday shopping season. Emarketer forecasts that Nov-Dec 2019 will witness ecommerce contributing to 42.8% of overall spending growth in the US, with mobile as the star of the show expected to bring in $64.29 billion. Moreover, with 6 fewer days of the holiday shopping season this year compared to 2018, your average user is just a click away from storming through those virtual gates on their favorite smartphone, and every mobile experience counts.

We at HeadSpin love seamless mobile experiences and great deals, and have put together a wishlist for our favorite shopping apps:

1.) One size doesn’t fit all -

- Test your app/website on all top devices in the market across Android, iOS and browsers in the real world. Your user may not always be accessing your products in the comfort of their home with uninterrupted WiFi, so account for all real world conditions.

- Don’t download images larger than the device resolution. This puts the burden on the client-side to download a bigger asset and resize it as well.  


  - Consider having a server-side ability to configure height and width of images via the URL parameters.
  - Perform lossless image compression on PNG and JPEG images. Consider using the WEBP image format, which provides great compression and image quality

2.) Your users like shopping directly from you, so make sure all traffic is encrypted to avoid MITM (Man In The Middle) attacks - everything should be TLS.

Source: Wikipedia

3.) No one likes waiting around when there’s a good sale. Any instances of low page content over 2 seconds should be eliminated in order to increase user engagement.

Flipkart was able to detect and fix performance bottlenecks in their code base and prevented two serious performance regressions from going live to millions of users. Using NimbleDroid, a HeadSpin company, Flipkart was able to pinpoint the issue and the app startup time was reduced by 45%.

4.) Your user is unlikely to buy the same outfit 10 times. Same goes for downloading duplicate content. Avoid instances of duplicate messages which indicate that the client received and downloaded the same resources multiple times.

5.) Bring cache only. Products images that are not seasonal or promotional should be cached on your CDN.

According to author & humorist Erma Bombeck, the odds of going to the store for a loaf of bread and coming out with only a loaf of bread are three billion to one. Your user is gearing up for your holiday sales, so check out HeadSpin to put your best mobile experience forward!

Appium Tools for Rapid Development of Functional and Performance Test

HeadSpin — Wed, 09 Oct 2019 20:35:20 +0000

Wednesday, Nov 5, 2019 | 9:00AM - 10:00AM (Pacific Standard Time) | 1 Hour

In this webinar we'll take a look at several tools we can use to
accelerate the development and debugging of Appium scripts for functional testing and performance testing.

Specifically, we’ll cover:

Overview of functional and performance testing
Appium Desktop, which is an all-in-one Appium server plus inspector tool. How to inspect and debug mobile web and hybrid apps.
web2driver, a browser-based Appium client which can be used as the foundation for a whole new category of tools.
The new HeadSpin Recorder, built on web2driver, which allows for rapid development of geo-distributed cross-device testing.
Appium’s Event Parser, which can be used to visualize the timeline of an Appium session to help discover bottlenecks.

About the Speaker

Johnathan Lipps - Architect and Project Lead, Appium

Jonathan is the architect and project lead for Appium, the popular open source automation framework. He is the founding Principal of Cloud Grey, a consulting firm devoted to helping clients leverage the power of Appium successfully, and is also an advisor to HeadSpin. He has worked as a programmer in tech startups for over 15 years, but is also passionate about academic discussion. Jonathan has master’s degrees in philosophy and linguistics, from Stanford and Oxford respectively. Living in Vancouver, he’s an avid musician, and also writes on the philosophy of technology.

What is a Mean Opinion Score or MOS?

HeadSpin — Thu, 12 Sep 2019 19:35:27 +0000

Mean opinion score (MOS) is a measurement of the quality of experience, commonly used in profiling the quality of audio and video content. HeadSpin’s AI engine continuously measures video mean opinion score of mobile application video during a performance session test. The following is a screenshot of Video Quality MOS on HeadSpin’s waterfall UI timeline highlighted with a red outline.

MOS Performance Metric in HeadSpin’s waterfall UI

Performance Session Link:

https://ui-dev.headspin.io/sessions/de8d3768-c57d-11e9-bcde-f01898ea5299/waterfall

Video contents are incorporated in different types of mobile applications: live streaming sports and events, live streaming news, explanation video, cutscene in a game, introduction/walkthrough video, live communication video, video as a file etc. Poor video experience can negatively impact user’s overall experience. It is crucial for mobile developers and QA teams to understand how well their video performs on real devices in real network conditions.

HeadSpin’s AI Engine MOS

HeadSpin’s mean opinion score ranges from 0 to 4, with 0 being very poor and 4 being excellent as shown in the following list.

Score range:

0 very poor
1 poor
2 fair
3 good
4 excellent

iPhone X Video Examples

MOS: 3.54 [Youtube video]

Comments: Although the content of the Chinese character and QR character are not crystal clear, no visible blockiness can be observed.

MOS: 2 [NBA Video with blur and blockiness]

Comments: Notice that the player on the video screen capture is blocky. The scoreboard on the video is blurry.

MOS Score: 0.57 [Youtube Video with severe blockiness]

Comments: Notice that the video is very blocky where the face of the person is indistinguishable.

Improve your video and delight your customers

With real networks on real devices, many factors can affect the delivery of video content through mobile applications. The following list includes some of these different factors:

The device and OS of the mobile application
Bandwidth, latency, jitter packet losses of the network
Misconfiguration of the content delivery network (CDN)
Mobile application client issue playing the video
Corrupted video download

Using HeadSpin’s platform video mean opinion score (MOS) and network performance capture, developers and QA teams can resolve issues and improve mobile video viewing experience, which ultimately leads to delighted customers.

HeadSpin enables you to test your mobile app on real devices and real networks around the world for real-world user experience conditions right from your web browser. No code changes needed. Start your Free Trial