DEV Community: Andre Vasconcelos

The rise of WebVR (and how Aframe can help fill the content drought)

Andre Vasconcelos — Mon, 15 Apr 2019 08:49:27 +0000

Introduction

Virtual reality has always been a topic of interest for the human population, as we've seen in movies, TV shows, books, and several forms of media that depict a distant future where reality itself has been replicated by technology, and that the bounds of what is possible can only be limited by one's own imagination.

While we may not yet be in the dystopian world of The Matrix, we've began paving our way to it with the recent advancements of VR technology.

Ever since Palmer Luckey re-ignited humanity's passion for the extraordinary with the introduction of Oculus' development kit 1, HTC has come up with its own line of head mounted displays (HMDs for short), and Microsoft created an ecosystem for Virtual Reality, to which companies like ACER, HP, and Lenovo have created different versions of HMDs to be run on that ecosystem.

This year is set to be an exciting year for VR, with the introduction of the Oculus Quest, a standalone 6DoF headset that doesn’t require a powerful gaming computer to run polished (albeit less graphically impressive) VR experiences, and new players like Valve introducing its own take on VR with a new generation of controllers that track the movement of every finger. We'll also be getting a number of high-quality games like Asgard's Wrath and a patch for No Man's Sky that enables full VR support.

As exciting as most of the things mentioned above are, Palmer Luckey himself recently wrote an article claiming that even if we were to distribute free VR headsets to the population worldwide, VR would still be hard pressed to succeed at keeping people engaged. But why is that?

The Content issue

Currently the majority of VR content is being delivered in the form of games. It makes sense given that many gamers aren't as price sensitive as the rest of the population, if it means that they get a premium experience out of the deal.

This becomes a problem, however, when trying to bring VR into the mainstream. Because even if we were to have cheap headsets available, there aren't many uses for them outside of gaming, where the non-gaming experiences currently available are limited to short stories, documentaries, or creative apps (3D modeling and drawing).

And when it comes to creating a full ecosystem of Virtual Reality applications, there are still many bases left to cover that are left open; some of them include:

Fitness apps
Educational apps
Social Media apps
Productivity apps
Shopping apps

Similarly to how the price of smartphones was justifiable because of the value brought to us by the App Store (or Google Play Store) and its diverse applications, VR will also require a certain amount of diversity and polish to its offerings in order to make the cost of a headset worthwhile.

The problem with that is that developing for VR is a costly, time-consuming process, and most of the people with experience in the disciplines required to create VR apps, were people who were already in the creative industry.

Enters WebVR and Aframe

While there's little doubt that currently the most polished and performant applications are built on top of the Unity or Unreal game engines, a new emerging alternative is being introduced, as modern browsers like Firefox (or browsers created from the ground-up with VR in mind, like Supermedium) begin to add support for the positional tracking of HMDs and controllers, as well as events triggered by these controllers.

The folks behind the previously mentioned Supermedium browser have created their own library that's been built on top of Three.js, called Aframe. Aframe enables you to create a 3D scene, import your own models, add objects and text, all with HTML and Javascript.

This is important because Aframe:

1. Lowers the technical barriers to entry

Developing for VR, as opposed to developing traditional websites, already include a plethora of new areas and topics to explore, such as creating and adding 3D models to your scene, adding sounds to enrich the experiences you're creating, and creating interfaces that are tailored for VR motion controllers.

While none of the above are trivial things to learn on their own, having to learn how to develop on Unity or Unreal engines (including brushing up your C# or C++ skills) on top of that makes it incredibly time-consuming and difficult for any given individual to get started on bringing their ideas to life.

At the very least with Aframe, you get to use your pre-existing knowledge of HTML and Javascript to be used as a foundation on which to build your ideas on top of. And with the many readily-made components, you can quickly put together a prototype, and work towards building a finished product.

But just how difficult is it to build a VR scene on Aframe?

<html>
  <head>
    <script src="https://aframe.io/releases/0.9.0/aframe.min.js"></script>
  </head>
  <body>
    <a-scene>
      <a-box position="-1 0.5 -3" rotation="0 45 0" color="#4CC3D9"></a-box>
      <a-sphere position="0 1.25 -5" radius="1.25" color="#EF2D5E"></a-sphere>
      <a-cylinder position="1 0.75 -3" radius="0.5" height="1.5" color="#FFC65D"></a-cylinder>
      <a-plane position="0 0 -4" rotation="-90 0 0" width="4" height="4" color="#7BC8A4"></a-plane>
      <a-sky color="#ECECEC"></a-sky>
    </a-scene>
  </body>
</html>

As seen above, a few lines is all it takes to create 3D objects and initialize a scene you can explore not only in VR, but also with your standard desktop or mobile browser.

2. Increases the diversity of creators

I'll admit, I'm making the assumption right now that most of the folks who can build experiences on Unity and Unreal engine are the ones who work within the entertainment industry, be it developing games, animations, or educational content that can be viewed in 3D. But I feel somewhat confident in saying that the people who have extensive amounts of experience in a specific industry, say, developing B2B productivity tools, will for the most part not be too familiar with game engines.

This means that the ease of development brought by Aframe enables people from various areas of expertise to try their hand at bringing over these experiences to Virtual Reality, and who knows what types of different ways to visualize information and interact with other people will now become commonplace inside VR?

Shortcomings of WebVR

As much as we see clear benefits and opportunities coming from consuming VR content through the web. There are still a few technical difficulties that should be overcome before WebVR can deliver contents of comparable quality when compared to game engines.

1. Size limitations

One of the main selling points of WebVR is the fact that no installation of a given app is required prior to using it.

However this can easily become one of its shortcomings given that assets still need to be downloaded onto the client, and depending on the size of your scene, you can still be better off going the traditional route by having users download and install your app with all its assets prior to loading up your experience, instead of having them download everything each time they log onto your website.

These limitations can be mitigated by caching assets that are frequently used, breaking down your scenes to use the minimum amount of resources possible (this is quite easily achievable when using route functionalities of Javascript frameworks), and offering the option of loading lower-quality assets (models with fewer poly counts, low-res textures and whatnot) to diminish download sizes for the client.

2. Performance limitations

VR is still in its infancy, and WebVR even more so.

This means means that large scenes can become unstable and suffer from frame-rate drops. And until WebVR matures further, scenes will have to be carefully constructed to be short and sweet, limiting poly counts on models and not relying on any over-the-top animations or particle effects.

This may be discouraging but let's remember just how ugly websites used to be in their early days before we got to the point where they can legitimately be considered works of art.

Conclusion

VR can be a beautiful medium for expression, and a powerful tool to get things done. Its potential, however, is not being thoroughly fulfilled given the lack of diversity and quality of experiences currently available in the market.

But I remain hopeful that WebVR can pave the way for developers of various backgrounds to create meaningful content and revolutionize their respective industries.

What do people think of ARKit?

Andre Vasconcelos — Mon, 17 Sep 2018 07:53:41 +0000

Has anyone been developing AR apps around here?
Was it easy to get into it?
Did you manage to scale it up well enough?
Does anyone have a good learning resource to recommend?

I personally have started a little while ago, and was surprised that while there are a lot of people writing articles about it, they're all covering fairly basic stuff and everything else I've been having to figure it out as I went (A lot of trials and failures ensued).

Rendering your first AR object with AWS Sumerian (2/2)

Andre Vasconcelos — Mon, 17 Sep 2018 05:27:32 +0000

If you've followed the previous article on this tutorial, you should now have a working project with two scenes attached to it.

We'll now go over how to create and publish scenes, and use these other scenes inside other scenes inside your projects.

Creating a 3D Entity

Adding an Entity

Now we're ready to keep going, click on the scene you want to move into and enter the editor screen

Since we just want to render a simple 3d object, this is what we're going to do:

Click on Create Entity at the top of the editor screen.
Select the Box entity on the top-left corner of the prompted screen.
(optional) Click on the box, and change its name property on the right-hand side of the screen, so that you can identify this entity more easily in the future.

As AWS Sumerian saves all changes automatically, we're pretty much done creating a basic 3D shape that would be rendered on the screen (Though you can feel free to make it as complex as you like).

Creating an asset pack

On the bottom left corner of the editor screen, you'll notice an Assets section in there with a variety of icons and a tree hierarchy menu.

That section is where you will be able to create Asset Packs, which can then be shared between all scenes in your AWS Projects. To add our own customized pack in there, follow these steps:

Click on the Create Pack button
Select the newly created New Pack
Change its name property on the right side of the screen, as this is a name you'll be referring back to in other scenes, you want to be able to recognize your assets
Drag the box entity on the top-left right of the screen (in the Entities panel) inside your new Asset Pack

If everything went well, we now have a custom pack with a 3D entity inside it, now all that remains in this step is to make it accessible to other scenes.

Exporting asset packs

To export any Packs you create, you just have to hover your mouse over the desired pack and click on the second icon to the left.

This should prompt a modal that will ask you what kind of pack you're going to be exporting, in this case we're exporting a pack in the Entities category.

Handling your AR Scene

Now that we have a readily available 3D asset on hand, we just have to enter our AR scene and make use of it!

Scene elements

Going back to the editor screen of our AR Scene. You'll notice on the left side of the screen that we have the following entities:

AR Camera: The initial point where the scene gets rendered, with the coordinates (0, 0, 0) on an (X, Y, Z) axis.
AR Anchor: An anchor is a coordinate you can use to attach entities to your scene, the AR Template comes with an anchor automatically placed at the (0, 0, 0) coordinate.

Importing assets

The first step we'll want to take is to import our asset pack to the scene.

Just click on Import Assets at the top, select the pack you want (notice that there are many pre-made options in there to play with as well!), and hit Add at the bottom of the menu. This should add a new pack on your Assets menu.

Adding a script

At this point we could just drag our 3D asset into the scene and render it in an app.

But I found that most real apps are more interested in programatically adding objects to their scenes instead of "hard coding" them in there. To accomplish this we'll be having to use AWS Sumerian's scripting functionality, which will require you to know a bit of Javascript.

First thing we want to do is adding a script to the ARAnchor entity:

Select the ARAnchor entity on the top-left tree hierarchy menu
Scroll down the properties pane on the right side of the screen, to find the Scripts section
Click the + button to add a new script, and select the Custom option

This will create a script template that contains a few life-cycle functions that are triggered at various times.

Over this example we'll only be working with the setup function, and adding our custom parameters at the bottom of the file.

Instantiating an object

We'll need to load our custom-made 3D asset into this script so that it can be rendered, for that we'll need to add a parameter for our function that can receive said asset:

var parameters = [
    {type: 'entity', key: 'exampleEntity', description: 'Example Entity'},
];

Note: You may call the key and description values whatever you wish, however it's recommended to keep it a value relevant to the asset pack you'll load into this parameter.

At this point you may save your script file, and you'll notice that a new slot with the Entity attribute appeared on the right-hand attribute menu (inside the Script section).

At this point, you can drag your asset pack into the empty slot and the name of your pack should now be seen inside that empty slot, and to finalize the process, add the bit of code that will instantiate your asset and place it inside the AR world scene:

function setup(args, ctx) {
    const arSystem = ctx.world.getSystem('ArSystem');

    // Creates an Entity and adds it to the world
    var x = 0;
    var y = 0;
    var z = -1.5;
    var translation = new sumerian.Matrix4(
        1, 0, 0, 0,
        0, 1, 0, 0,
        0, 0, 1, 0,
        x, y, z, 1,
    );

    var clonedEntity = sumerian.EntityUtils.clone(ctx.world, args.exampleEntity);
    clonedEntity.addToWorld();
    clonedEntity.setComponent(new sumerian.ArAnchorComponent());
    arSystem.registerAnchor(translation, function(anchorId) {
        clonedEntity.getComponent('ArAnchorComponent').anchorId = anchorId;
    })
}

To explain the code seen above:
Firstly we instantiate ArSystem, a built-in functionality in Sumerian that allows the user to perform some functionality native to AR. View documentation here.

After that we set the x, y, and z values that we want our 3D entity to be rendered at, and create a transform value from them to be used later.

We then use the EntityUtils to generate a clone of our exampleEntity, which we can receive from the args parameter, and add it to the World.

As AR functions around tying Anchors to virtual objects, we now need to take the extra steps of attaching an Anchor component to our clonedEntity, and then register a new anchor from the tranlation values we set earlier.

Lastly we set the anchorId value of our clonedEntity to match the one we've just created, which will result in your object to be viewed at the position set on your translation value.

Loading the scene in an iOS App

Now that we've set up all the logic behind instantiating nodes and anchors, as well as rendering a 3D entity in real-space. We now have to test that with a real device.

The first thing you want to do is Publish your scene using the top menu of the editor screen, or hitting Shift + Ctrl + P. Then you may follow these steps to
run it on your device:

Running on iOS

Clone the SumerianARKitStarter repository.
Open XCode, choose Open on the File menu, and then navigate to the SumerianArKitStarter.xcodeproj file in the top-level directory of the cloned repository.
Set up your code-signing identity and provisioning profile on the General tab of the application settings. See Apple’s documentation if you’re not familiar with these concepts.
In XCode, open the ViewController.swift file. Change the sceneURL variable to point to the URL of your published scene from the previous step. Append /?arMode=true to the URL. This parameter causes the scene to render in a way that’s more amenable to an AR experience (specifically, it enables the alpha channel on the WebGL canvas and removes the UI elements).
Connect your iOS device. Build and run the application.

Running on Android:

Clone the SumerianARCoreStarter repository.
Open Android Studio, choose Open an existing Android Studio project from the menu, and then navigate to the SumerianARCoreStarter directory in the top-level directory of the cloned repository.
In Android Studio, open the MainActivity.java file. Change the SCENE_URL variable to point to the URL of your published scene from the previous step. Append /?arMode=true to the URL. This parameter causes the scene to render in a way that’s more amenable to an AR experience (specifically, it enables the alpha channel on the WebGL canvas and removes the UI elements).
Connect your Android device. Build and run the app.

Wrapping up

Hopefully by following all of these, you've now been able to set yourself up with an AWS Sumerian project and you've grasped the basics of how to import your own elements of a scene, and make them appear on your screen on both iOS and Android devices.

While still in its infancy, the possibilities of hosting AR scenes on the web and having it be adaptable for all types of mobile devices is very exciting for me, and although this was a very shallow view of what you can do with AWS Sumerian, I invite you all to join the AWS Sumerian Slack page and become a part of this exciting initiative.

My apologies if the writing of this tutorial was too messy, I'm not yet very experienced with writing articles like this, but I hope that I can improve with practice :)

References:

Rendering your first AR object with AWS Sumerian (1/2)

Andre Vasconcelos — Mon, 17 Sep 2018 05:26:31 +0000

What is AWS Sumerian?

As we've seen the introduction of Apple's ARKit, and the somewhat recent release of Google's ARCore, we're getting to see a whole new generation of apps that incorporate Augmented Reality into its stack of features.

Amazon couldn't help but join the race by providing a different way in which to allow developers to create an AR environment. While in its present state developers have to create scenes into their respective applications (be it with ARKit, or ARCore), AWS has provided us with a separate option of publishing, and loading our scenes directly from the internet.

This means that as long as you're willing to compromise and have your AR experience be dependent on having an internet connection (which may be a fairly big barrier to some), you can develop a single AR scene that can be quickly incorporated into native ARKit and ARCore applications.

In this series of articles we'll be viewing how to get started with an AR Application, then we'll look at how to add your own custom Entities to your scenes, and add a layer of interactivity to them utilizing state managements and AWS Sumerian's scripts.

Requirements

An active AWS Account

Getting Started

To begin, head over to your AWS Console, and under the VR & AR section, you'll want to click on Amazon Sumerian. This will prompt a new tab to launch and you should be sent over to your Dashboard, now we're ready to begin!

1. Creating Your Project

Now that we're ready, notice how your screen is divided into 3 main sections. To your left you have your navigation menu, where you're able to create new projects to manage them under, in the center you'll notice a big Create New Scene button, but let's not click on that one yet! And on the right section you have a sidebar that will throw you information about whatever you selected in the other sections

What we want to start with, is to create a new project where we'll be handling all scenes and entities in our tutorial series. We'll do that by:

Click on the Projects option on our navigation menu
Click on New Project
Input Practice Sumerian Project, or whatever your heart desires
Hit the Create button

IMPORTANT: In order to be able to import a 3D asset into your AR Scene, they will need to be a part of the same project

2. Creating your scenes

Now that we have a project set up, we'll be wanting to create the following scenes:

A 3D entity to render in AR
The AR scene itself

3D Entity Scene

Start by going back to the Home of your dashboard
Click on Create New Scene. AWS Sumerian will prompt you for a name, so we'll input example_entity
Hit Create

If everything goes as expected, we'll now be sent to the editor screen with an empty scene.

BUT WAIT, THERE'S MORE!

In Sumerian's current version, you can't add a scene to a project at the moment it's created. To do that you'll have to do the following:

Return to your Dashboard by clicking on the top-left Scene dropdown menu, and clicking on Exit to dashboard....
Click on the scene you just created (be sure not to click on the bottom part of it, as you'll be sent directly to the editor screen)
On the right sidebar, look under the Actions section, and select the Move option
Click on the project you just created and hit Move

phew!
Now for the last part of setting our project up for AR!

AR Scene

We're going to create our AR scene using the following steps:

Go over to Home in your dashboard, and look down at the Create scene from template section in the middle.
You'll notice we have an Augmented Reality template all ready for us. Click on that, input a name for your scene and Create it.
Just as the previous step, you can go back to the dashboard and Move this scene into your project

Wrapping up

If you've completed all the things described above you should have:

One practice project where you can manage all your scenes and assets in.
One scene where you will be creating a 3D entity
One AR scene where we'll import 3D assets into

You may now proceed to the second part of this tutorial, where we'll go over how to create 3D entities and programatically add them into AR scenes using AWS Sumerian's scripts.

Bonus information

AWS Sumerian community on Slack: https://amazonsumerian.slack.com
Official link to tutorials on AWS Sumerian: https://docs.sumerian.amazonaws.com/
Twitch tutorials on how to work with Sumerian: https://www.twitch.tv/jsmeester/videos/all