DEV Community: Nino Ross Rodriguez

Vibe Coding a Web Game from Scratch — Fast, Fun, and Occasionally on Fire

Nino Ross Rodriguez — Tue, 14 Jul 2026 13:00:00 +0000

Introduction

Vibe coding is the art of building software purely from instinct and creative flow. No wireframes. No JIRA tickets. Just “trusting the Force” with an AI platform like ChatGPT or Copilot. Vibe coding is what happens when you stop asking and thinking “how do I build this properly?” and start asking “what feels right right now?”

It’s fast, addictive, and when things click, it delivers instant dopamine. But it can get messy — really messy — real quick.

Why I Vibe Coded a Whole Game

My son and I “invented” a new card game for us to play and we had a lightbulb moment. “This could potentially be a game for everyone could enjoy.” I wanted to reach a large audience without shelling out money to get it tested and decided first to create a digital prototype. Since vibe coding is the trending topic in the industry today, it was a perfect opportunity for me to see what the fuss is all about.

What followed was a streak of successful prompts — designing the UI, shaping the mechanics, fixing things on the fly. I was possessed, tweaking everything as I went, solving UX problems as they appeared in front of me.

Most of it worked. Then disaster and frustration crept in. I did manage to get to a working prototype that I am happy with, and learned a lot about what vibe coding is, isn’t and where it can take you.

Elementara: Spirit of the Dice — The Web-based Project

Elementara is a 1v1 dice-based creature battler. You control a squad of three elementals, fighting until the last creature stands. Combat is determined by dice rolls — misses, hits, criticals — and each elemental has unique strengths and resistances. I added a twist: the ‘Corrupted Dice’ mechanic, which buffs a player after they roll the same result twice in a row. It adds an element of surprise into what could’ve been a pure RNG (random number generator) slugfest.

This sounded like a complex game that would take up weeks, probably months to get a working prototype up and running. I felt that this is the perfect testing ground to see if I can build the entire project in half the time. I wanted to see if I could build it in a fraction of the time a typical dev team would, push generative AI to the limits and see what it can do and debunk the myth “will generative AI replace developers?”

The Vibe Coding Process

I was hyped up, ready to start with the project. After all, it’s my first game that I would be developing and I would be vibe coding through the entire process. The vision was clear as well, “I want my game to be easy to use by many, intuitive and fun to play with for all ages.” The tools were simple: generative AI. ChatGPT for ideation, Copilot for illustrations, and a mix of v0 and Copilot for code generation.

ChatGPT — The Game Designer

ChatGPT was my best friend from the start. I gave it specific instructions on how I could integrate a card-battle game with a dice and we managed to come up with the idea of using a 6-sided dice to be the decider of the damage, somewhat similar to Dungeons and Dragons. ChatGPT also helped me come up with the names of the elementals, their hit points, strengths and weaknesses. It also helped me polish the game mechanics and rules to make the game more balanced without sacrificing the “fun” aspect of the game.

Copilot (Image Gen) — The Art Department

Amongst all the image generative AI platforms available in the market, Copilot was the only one that managed to please me with its output. I gave ChatGPT, Copilot and Gemini the equal opportunity to create images based on my prompts and very specific art style. Copilot was the only platform that gave me images that are as close as possible to the art style I wanted for the game. It was the only platform that has provided me with a lot of generated images before hitting its limits, unlike ChatGPT which maxes out at 3–4 images.

v0 & Copilot (Code) — The Engineers

Vibe coding with AI code tools was tricky but effective — until it wasn’t.

v0 was amazing at scaffolding the app based on prompts. But because of the free tier, I had to be precise with my requests. I started by feeding it the rulebook and watching the UI and game flow emerge. It worked — mostly.

Copilot, integrated in VSCode, filled in the gaps. It helped tweak logic and connect components, especially once v0 started falling apart. v0 couldn’t determine which component I was pertaining to and ended up changing the entire UI of the game.

Key Findings & Lessons Learned

Most of my coding decisions were based on instincts. I eyeballed bugs, adjusted UI issues immediately, and kept rolling. But I had to constantly remind myself: be very specific with prompts, or AI will interpret things however it wants.

In an accumulation of roughly 40–50 hours of development, I managed to wrap up Elementara in a nice red ribbon. I was surprised and very satisfied with the results where it was 90% accurate most of the time and provided to me in a few seconds.

After a few hundred prompts and getting the game to 75% completion, I started to receive hallucinations from AI and frustrations build up quickly. The main issue that annoyed me was that v0 started to change more of the UI when I specifically told it not to. Another issue was that the platform started to “stall” and wrote large parts of the codebase arbitrarily, no matter how detailed my prompt was. Lastly, v0 could not pinpoint the main issue that I wanted to get fixed. The last 25% of the project involved digging into the code myself and fine-tuning everything manually.

As much as I enjoyed vibe coding through this side project, it is not recommended for any production work. It can help your team to get up to speed in prototyping a single component. But beyond that needs manual intervention and developer experience in fine tuning to the requirements. I have not yet explored how to extend the game or whether the code is structured properly. I am sure though that I will be opening a can of worms when I do.

Conclusion

If you’re a solo dev, indie creator, or just trying to ship something fun — vibe coding is a legit weapon. Generative AI can get you from idea to prototype faster than any team ever could.

But if you’re in production? Managing a team? Building long-term?

It’s a trap.

AI can scaffold code fast — but it still needs human judgment. Left unchecked, it breeds technical debt, unstable logic, and brittle foundations. The deeper you go, the messier it gets.

Use it. Abuse it. Just don’t trust it blindly.

Vibe coding will leave you with:

Unstable code
Hidden tech debt
A fragile house of cards

My take:

Use it for creative prototyping and fast MVPs
Expect to rewrite large chunks of AI-generated code
Don’t trust the vibes — review every line

Before I go, here’s the game that I vibe coded. Let me know your thoughts in the comments.

elementara.vercel.app

https://elementara.vercel.app/

Vibe Coding: Fast, Loose, and Headed for Trouble

Nino Ross Rodriguez — Tue, 07 Jul 2026 13:00:00 +0000

In the age of generative AI and auto-complete-everything, a new pattern is emerging across the web industry: vibe coding.

It’s like using the Force — you close your eyes, trust your instincts (or your AI), and just build. A few sentences to an AI model and a sense of direction.

It sounds cool — even powerful. For many, it is. But behind the wizardry is a serious question: are we building smarter and faster, or just drifting closer to the dark side of technical debt?

What Is Vibe Coding?

Vibe coding is the practice of writing or shipping code based more on instinct, AI suggestions, and or “what feels correct” than on understanding the entire code, or formal architecture. It’s the “trust the Force” approach to software development — you’re feeling your way through the flow, trusting that the tools (and your gut) will guide you to the right outcome.

This coding style often thrives in high-speed environments: prototypes, hackathons, MVPs, startup scrambles, or AI-assisted workflows where code suggestions come faster than your brain can vet them.

The term vibe coding draws from two key realities. First, developers are increasingly trusting AI-generated code without fully understanding how it works — or what it’s doing under the hood. Second, non-developers are now building with low-code tools and AI-generated components — often without any visibility into the underlying logic or architecture.

Why It’s Taking Off

This shift toward vibe coding has real appeal. Teams appear more productive, products are shipped faster, and stakeholders get what they want — quickly. AI copilots are now “good enough” to scaffold working solutions in minutes — drastically reducing the effort needed to get something up and running. At the same time, time-to-market pressure often prioritises output over long-term maintainability. Even junior developers can move fast — often bypassing traditional engineering mentorship, best practices, or architectural guidance. But like most shortcuts, all this speed comes at a cost.

How Vibe Coding Impacts Teams (and Products)

For developers, vibe coding can feel like a superpower. It enables fast feedback loops and rapid shipping, making it ideal for quick iteration and exploration. But that speed often comes at the cost of structure. AI-generated or improvised code is frequently unoptimised, difficult to read, and even harder to scale or maintain — making it a breeding ground for technical debt. Developers may also become overly reliant on their tools, deploying code they don’t fully understand. Debugging turns into a painful guessing game when things break.

For non-developers, vibe coding lowers the barrier to entry. It empowers anyone to “build” a product, regardless of technical background. But that accessibility hides a dangerous tradeoff — what appears simple on the surface may be fragile and complex underneath. Without proper guardrails like documentation or architecture, these systems quickly fall into pieces into dependency hell — where one small change risks breaking the entire project.

How Vibe Coding Creates Technical Debt

Technical debt isn’t just about messy code — it’s about turning today’s shortcuts into tomorrow’s roadblocks. With vibe coding, this debt tends to accumulate quietly — fast. Teams often skip writing tests, neglect documentation, ignore naming conventions, and overlook scalability. Worst of all, they may not fully understand the code they’re deploying. Integrations may work initially — but are fragile and prone to breaking as the product evolves. By the time someone decides it’s time to “clean it up,” the damage is usually baked in. Rewriting or refactoring ends up being far more expensive than doing it right in the first place.

Cheap Vibe Coder or Expensive Experienced Developer?

At some point, every decision maker hits this crossroads — do you hire a cheap vibe coder who can move fast and make something “work” now, or invest in an experienced developer who’ll build it right from the start?

It’s a tempting tradeoff. The vibe coder promises speed, lower cost, and a working MVP by the end of the week. They’ll probably deliver something that looks functional — on the surface.

But under the hood, the risks stack up: no real architecture, no scalability plan, and no thought given to long-term maintenance. As the product grows, so does the pain — slower iterations, unexpected bugs, rework, and in many cases, a complete rewrite.

On the other hand, experienced developers don’t just write code — they think through edge cases, plan for growth, and build with the future in mind. It might take a little longer and cost a bit more up front, but it pays off in reduced risk, smoother scaling, and less firefighting down the road.

It’s not just a talent choice — it’s a strategy call. Do you want to move fast today and pay the price later, or build a foundation that won’t crack under pressure?

Recommendations for Decision Makers

You don’t need to kill the vibe — but you absolutely need to manage it.

Vibe coding has its place. It’s excellent for prototyping, quick iterations, and getting ideas off the ground fast. But it’s not suited for scaling mission-critical systems — where stability, clarity, and long-term maintainability matter.

To keep things sustainable, teams need to invest in cleanup. That means allocating time for regular refactoring and code audits — treating them not as afterthoughts, but as part of the product’s hygiene.

It’s also essential to establish guardrails. Linting, testing, and code reviews shouldn’t be optional — even when working with AI-generated snippets or low-code tools. These practices create structure and catch problems early.

Finally, balance is key. Pair your fast-moving builders with experienced system thinkers, architects — who can see the long-term implications and ensure the foundation remains solid as the product evolves.

Final Thought

Vibe coding is here — and it’s not going away. The tools are getting better. The pressure to ship is only increasing. The opportunity lies in treating vibe coding as a tool — not a development philosophy.

Left unchecked, it can rot your codebase from the inside out. But with the right oversight, vibe coding can be your team’s creative engine — with the brakes, seatbelts, and map included.

An Open Letter to Everyone Who Thinks “Vibe Coding” Will Replace Developers

Nino Ross Rodriguez — Tue, 30 Jun 2026 13:00:00 +0000

Dear Prompt Warriors,

So you typed, “Create me a bold and minimalist UI,” and BOOM! A flashy interface appears. You just spun up a page without writing a single line of code. No VSCode, no React boilerplate, no debates about BEM naming or accessibility. Just pure, effortless generation.

Now you’re thinking: Why would we ever need developers again?
I get it. You feel empowered. It feels like magic. I’ve been there. But let me break something to you gently, or maybe bluntly:
Vibe coding isn’t coding. It’s cosplay.

Sure, AI can generate a UI that looks great at a glance, in a fraction of the time it would take a dev. It might even pass as an MVP on a pitch deck. But underneath that shiny surface? It’s duct tape. No scalability. No reusable components. No accessibility. No fallbacks. No security.
You’re not building a product, you’re building an illusion.

And the moment it needs to be maintained, extended, debugged, or deployed? You’re going to need someone who knows what every single line of code actually does, not someone who just says, “Make it snappy with a Gen Z feel.”

Let me paint you a picture of what that duct-tape codebase really looks like:

Phase 1: Empowerment
I created a web-based game entirely through vibe coding. Hit after hit of dopamine. It felt like I was building something amazing. I powered through. UI? Done. All that was left were the game mechanics. Easy, right?

Phase 2: Frustration
At the tail end of the project, I was ready to deploy and share it with testers. But after 200+ prompts and reverts, it still wasn’t usable. I faced regression issues, random “features” I never asked for, and the best part? The AI rewrote the entire codebase, nuking all the UI and functionality. Amazing.

Phase 3: Intervention
Let’s say I somehow made it to the finish line, a working game, ready to ship, just needing layout and copy tweaks. I cloned the codebase locally for faster iteration. Yes, the folders were neatly stacked, just as I instructed. But then I opened the file that contained the core game logic.

Thousands of lines. All in one file.

No separation of logic, no type safety, no modular flow. Digging deeper, I found repeating blocks, hardcoded logic, and zero components. I was trying to change a one-liner but had to sift through a dumpster fire of code.
A fresh grad would’ve done a cleaner job. How am I supposed to scale or refactor this?

If you believe vibe coding will replace developers, you’re not a developer. You’ve never read the code it generated. You’ve never tried to fix a bug, or maintain the thing it built. You’ve never stared down a thousand-line JS file while trying to debug a div.

Vibe coding is great for mockups, prototypes, and play. It’s perfect when you need fast visuals for a pitch. But when it’s time to ship a real product? You’ll need more than vibes and prompts.

You’ll need us.

Developers

P.S.

DM me when you’re ready to build something real, together.

If AI Replaces Juniors, Who Replaces Seniors? The Hidden Cost of Misusing AI in Tech

Nino Ross Rodriguez — Tue, 23 Jun 2026 13:00:00 +0000

The hype of AI is at its peak. Scroll through LinkedIn and Medium and you will see “vibe coders” claiming that programming is dead. Startups pitch to investors on products that were “built entirely with AI.” Companies whisper in boardrooms that developers are replaceable, cutting cost and time. The hype has twisted the industry’s perception on AI. It is seen less as a tool and more as a replacement.

That’s not just wrong, it’s dangerous.

The Illusion of AI as a Replacement

Hollywood might be the first one to plant the seed in our heads that AI will one day take over us. We are not there yet, but AI can spit out code, debug snippets, or scaffold an app faster than a junior and even a senior developer. On the surface, it looks like a replacement. But here’s the catch: generating code is not the same as building, maintaining a product and making critical decisions.

Real software development is more than writing lines of neat code. It’s:

Understanding business logic
Architecting systems that scale
Navigating security, compliance, and integrations
Maintaining legacy code and evolving it over time

AI is a pattern machine. It references from existing sources found in the internet and learns from human input. But, it doesn’t understand if the suggestion is relevant and accurate. It doesn’t make trade-offs. It doesn’t negotiate with stakeholders or know why one solution is better than another.

Treating AI like a self-sufficient developer is like thinking a scientific calculator can replace a mathematician.

The Productivity Multiplier We’re Ignoring

The real promise of AI isn’t replacement, it’s augmentation. Developers can leverage AI by automating boilerplate and repetitive work; speed up debugging and testing; and focus on more important problems rather than bogging down on syntax.

It’s the same story as every technological leap. Spreadsheets didn’t eliminate accountants. CAD didn’t eliminate architects. They amplified what humans could do. AI should be the same for developers, if we use it right.

The “No Juniors” Trap It’s in the headlines everywhere.

“AI threatens entry-level jobs as university grads struggle to get hired”

“AI-driven layoffs are shrinking the job market for recent grads”

“‘A black hole’: New graduates discover a dismal job market”

If companies cut junior developer roles because “AI can do it cheaper and faster,” they destroy the pipeline of future talent.

Picture this:

In 2025, a product company replaces its junior devs with AI copilots
By 2030, their seniors start leaving, retiring, or burning out
Who will maintain the legacy systems? Who will inherit the institutional knowledge?
AI keeps patching code, but without understanding the why, and the technical debt piles up
Hiring becomes a nightmare because there’s no generation of mid-level devs to replace the seniors

The chain has been broken, the company will be left dangling.

The Robot Fallacy

The industry keeps projecting sci-fi fantasies onto AI. We think of it as a self-aware robot that can “just do things” autonomously. Deploy it, press the red button and away it goes 24/7. But AI today is context-blind. It can autocomplete based on what you are typing, but it can’t architect on an optimal solution. It can generate lines of code within seconds, but it can’t govern what is happening under the hood.

Until that changes, and we’re not even close to it, replacing humans outright is reckless.

We’ve Been Using AI Wrong

The hype has blinded us. It may even brainwashed us all. The conversation keeps circling around “AI replacing developers” when it should be “AI helping developers.” AI is not here to replace the people who build, maintain, and evolve systems. It’s here to make those people more productive, just like every tool before it.

The companies that understand this will thrive. The ones that don’t will eventually collapse under the weight of their own short-sightedness.

AI is your fast thinking and working assistant, not an self-aware architect. A tool, not a team. If we keep misusing it as a replacement instead of a force multiplier, we won’t just harm developers. We’ll cripple the very industry we’re trying to “revolutionise.”

AI won’t replace developers. But companies that misuse AI will replace themselves, out of existence.

Laws of UX in AR

Nino Ross Rodriguez — Tue, 12 Nov 2019 23:15:01 +0000

[originally posted at infinite imaginations]

As technology is improving and getting more affordable, Augmented Reality (AR) has been constantly gaining popularity. From enterprise products like Google Glass Enterprise Edition 2, Microsoft HoloLens 2 and Magic Leap, to consumer available devices like AR-ready mobile phones and browsers, it has provided users with an immersive experience through the use of AR. But at what cost? Does having an Ironman-esque interface really provide a better experience? Is AR the solution for everything since it's trending?

The State of AR Today

From the word 'augmented,' a device will project or embed a digital object or information on the screen, simulating as if it was in the real world. AR comes in different forms and sizes but most of them have a few things in common:

Devices must be AR ready in order to 'project' the digital object on a real environment or object.
May need markers or trigger spots in order for the digital items to be embedded into reality.
Apps may need to be downloaded for the digital items to be embedded into reality without the use of markers.

Regardless of which form AR takes, developers have already explored and played around with the technology. Most of which are extremely brilliant ideas and were executed excellently but somehow has forgotten about the user experience side of things. At this point in time, AR is more of a tool and or a gimmick for most users. You will get users to use AR at most 5 minutes, 10 might be a stretch if they are playing an AR game like Pokemon Go. Players even have disabled the AR feature of the game to save battery.

London Underground Map Concept

How #AR transforms a simple train ticket into a transportation guide pic.twitter.com/SEEBKXt4Ig

— Alexandre Dip Hannemann (@DipAlexandre) May 30, 2019

At first glance, it is a brilliant concept displaying information about the local subway lines and the stops of different tracks. The developer can even extend it to display where the passenger is currently and how to get to their destination. However, one has to consider the barrier of entry on how passengers would get to use this AR feature. The passenger must first and foremost have the app to display AR. Another requirement would be an internet connection, most underground subways lack a good internet connection. These could be factors on hindering passengers to use the AR feature.

Philips Hue Concept

#AR s true potential is in blending interactions with physical and virtual things. Here is an #ARKit experiment to control #PhilipsHue made with #Swift on #iOS. Excited to see what #WWDC19 has to offer in ARKit pic.twitter.com/4vHkvdqdIG

— Sarang Borude (@doomdave) June 2, 2019

Another brilliant concept on connecting the physical and virtual world by creating swatches in the digital world, to control and change the color of smart lights such as a Philips Hue lightbulb. It is a nice idea to play around with mobile devices and see physical objects change to your liking. However, the idea and the need of pointing a mobile device on a smart light just to change the colour and intensity of the light is quite silly. The user just needs to change the settings with minimal swipes and or taps.

Restaurant Menu Concept

📱🍷AR Restaurant Menu Concept

Finally discover what the dishes really look like and check out their reviews!

I created this with ARKit 2.0.#ARKit #AugmentedReality #iOS12 pic.twitter.com/A1HfTmH1EF

— Oscar Falmer #AR (@OsFalmer) August 6, 2018

The idea is good and well executed. People do want to know what they are paying for in restaurants. Why not visualise the food first before buying it and look at the reviews? However, the user still needs to download the app and must be connected to the internet to view the virtual menu. This could be a barrier of entry, especially for those who are just hungry and would like to order food. It’s quite silly to hold the menu and your mobile device to see the virtual menu.

AR Done Right

Not all AR projects are and will be successful despite the trending technology. There are a few critical items to consider to have an effective AR project.

Solving a critical issue/problem

Trying to be the Swiss Army Knife of AR could be the downfall of the project. The idea of creating an AR project should be trying to solve a problem, making a task easier, or have a simple goal for the users.

Barrier of entry

Because most AR projects require apps to functionally run perfectly, think of how to get the users to download the app or get to the AR project. There are a lot of ways to onboard users but not all of it would be the right solution. The most simplest and most common barrier of entry is the use of QR codes.

Ease of use

If a user has passed the initial barrier of entry, the AR project must be easy to use and easy to understand. Not only that the project must intuitively guide the users how to use AR, but must be able to use it with minimal instructions due to the screen real estates are quite limited. Bombarding a mobile device screen with icons and instructions on how to use the feature can discourage the user from using the app.

Information overload

This is the common pitfall for any AR project. Designers and developers may at times have gone overkill and just have dumped everything on the screen. Screen real estate is quite limited and trying to bombard the users with information could be another hindrance of the project. Only show the needed elements and or information needed on the screen.

UX and AR Working Together

There is a collection of principles that designers and developers can refer to when designing and developing user interfaces for AR called the Laws of UX. I have chosen a few of them that can be applicable for AR.

Aesthetic Usability Effect

'Users often perceive aesthetically pleasing design as design that’s more usable.'

With a small screen real estate on mobile devices, designers have to consider how the AR project is going to work easily and intuitively for the user. One has to consider the design of the user interface and how the elements can easily portray instructions to the user without a lot of external interventions. There is a mentality that if the design is poorly done, the quality of work is directly proportional to the quality of usability and content of the product. In addition to this, AR has been interconnected with future technology. It is somewhat expected that users will be 'awed' with the AR projections.

Doherty Threshold

'Productivity soars when a computer and its users interact at a pace (<400ms) that ensures that neither has to wait on the other.'

In this day of technology, users have a shorter attention span and patience. Now that 5G connection is being rolled out, connection speed will be faster as users’ expectations will increase also. It is critical once the project has attracted a user’s attention, the project must be able to deliver information as soon as possible. A possible barrier here would be the ability of the user to interact with the AR, downloading, installing and or opening AR as quick as possible.

Hick’s Law

'The time it takes to make a decision increases with the number and complexity of choices.'

We all have seen futuristic interfaces in science fiction movies like Oblivion, Iron Man and Robocop. We all admit that it does look impressive and sleek to see a lot of elements flying around the screen. But take note: your users does not have a high processing brain to accommodate all the information on the screen. Leave the fancy user interfaces to the special effects team in movies. Instead, provide a simple and clean interface for the user to easily digest. The lesser choices the user has to make, the easier it will be for them to get on board and start using your project.

Miller’s Law

'The average person can only keep 7 (plus or minus 2) items in their working memory.'

In conjunction with the other laws of UX, Miller’s Law summarises how an interface and or user journey should be kept simple. By freeing the users’ mind from complex tasks and user journeys, this will enable them to achieve and do more with the project. Having a very long on-boarding process for example could discourage any user from continuing forward. Specially with AR, where the time from downloading the app to the time where the user can finally use it is much longer than the time of actually using it.

ARe you ready for the Future?

Though the technology may have arrived at the comfort of our mobile devices, there is still a niche market for AR. Not every AR project will satisfy the consumers’ wants and needs and we are still far from the reality of achieving the full potential of AR due to technological restrictions. While AR has landed a special place in education, retail and training, it is slowly making its way through the web with the help of developers. Experimental projects has been circulating around the internet to showcase the possibilities of Web AR. Where there is web, there is the mobile web. And where there is the mobile web, there is the potential of tapping millions of users.

As a food for thought in conclusion, there has been some technological advancements (VR and AR in the Mobile Web) and some are even coming back (Are QR Codes Making A Comeback?). I believe that AR can have a wider audience reach by leveraging these technological advancements in which can help decrease the barrier of entry for users. By keeping in mind the Laws of UX, that are not only applicable for design, it can help also help your AR project provide a smoother and immersive experience.

Using Artificial Intelligence to Generate Alt Text on Images

Nino Ross Rodriguez — Sun, 03 Feb 2019 19:51:25 +0000

Web developers and content editors alike often forget or ignore one of the most important parts of making a website accessible and SEO performant: image alt text. You know, that seemingly small image attribute that describes an image:

<img src='/cute/sloth/image.jpg' alt='A brown baby sloth staring straight into the camera with a tongue sticking out.' >

📷 Credit: Huffington Post

If you regularly publish content on the web, then you know it can be tedious trying to come up with descriptive text. Sure, 5-10 images is doable. But what if we are talking about hundreds or thousands of images? Do you have the resources for that?

Let’s look at some possibilities for automatically generating alt text for images with the use of computer vision and image recognition services from the likes Google, IBM, and Microsoft. They have the resources!

Reminder: What is alt text good for?

Often overlooked during web development and content entry, the alt attribute is a small bit of HTML code that describes an image that appears on a page. It’s so inconspicuous that it may not appear to have any impact on the average user, but it has very important uses indeed:

Web Accessibility for Screen Readers: Imagine a page with lots of images and not a single one contains alt text. A user surfing in using a screen reader would only hear the word “image” blurted out and that’s not very helpful. Great, there’s an image, but what is it? Including alt enables screen readers to help the visually impaired “see” what’s there and have a better understanding of the content of the page. They say a picture is worth a thousand words — that’s a thousand words of context a user could be missing.
Display text if an image does not load: The World Wide Web seems infallible and, like New York City, that it never sleeps, but flaky and faulty connections are a real thing and, if that happens, well, images tend not to load properly and “break.” Alt text is a safeguard in that it displays on the page in place of where the “broken” image is, providing users with content as a fallback.
SEO performance: Alt text on images contributes to SEO performance as well. Though it doesn’t exactly help a site or page skyrocket to the top of the search results, it is one factor to keep in mind for SEO performance.

Knowing how important these things are, hopefully you’ll be able to include proper alt text during development and content entry. But are your archives in good shape? Trying to come up with a detailed description for a large backlog of images can be a daunting task, especially if you’re working on tight deadlines or have to squeeze it in between other projects.

What if there was a way to apply alt text as an image is uploaded? And! What if there was a way to check the page for missing alt tags and automagically fill them in for us?

There are available solutions!

Computer vision (or image recognition) has actually been offered for quite some time now. Companies like Google, IBM and Microsoft have their own APIs publicly available so that developers can tap into those capabilities and use them to identify images as well as the content in them.

There are developers who have already utilized these services and created their own plugins to generate alt text. Take Sarah Drasner’s generator, for example, which demonstrates how Azure’s Computer Vision API can be used to create alt text for any image via upload or URL. Pretty awesome!

See the Pen Dynamically Generated Alt Text with Azure's Computer Vision API by Sarah Drasner (@sdras)on CodePen.

There’s also Automatic Alternative Text by Jacob Peattie, which is a WordPress plugin that uses the same Computer Vision API. It’s basically an addition to the workflow that allows the user to upload an image and generated alt text automatically.

Tools like these generally help speed-up the process of content management, editing and maintenance. Even the effort of thinking of a descriptive text has been minimized and passed to the machine!

Getting Your Hands Dirty With AI

I have managed to have played around with a few AI services and am confident in saying that Microsoft Azure’s Computer Vision produces the best results. The services offered by Google and IBM certainly have their perks and can still identify images and proper results, but Microsoft’s is so good and so accurate that it’s not worth settling for something else, at least in my opinion.

Creating your own image recognition plugin is pretty straightforward. First, head down to Microsoft Azure Computer Vision. You’ll need to login or create an account in order to grab an API key for the plugin.

Once you’re on the dashboard, search and select Computer Vision and fill in the necessary details.

Wait for the platform to finish spinning up an instance of your computer vision. The API keys for development will be available once it’s done.

Let the interesting and tricky parts begin! I will be using vanilla JavaScript for the sake of demonstration. For other languages, you can check out the documentation. Below is a straight-up copy and paste of the code and you can use to replace the placeholders.

var request = new XMLHttpRequest();  
 request.open('POST', 'https://[LOCATION]/vision/v1.0/describe?maxCandidates=1&language=en', true);  
 request.setRequestHeader('Content-Type', 'application/json');  
 request.setRequestHeader('Ocp-Apim-Subscription-Key', '[SUBSCRIPTION_KEY]');  
 request.send(JSON.stringify({ 'url': '[IMAGE_URL]' }));  
 request.onload = function () {  
     var resp = request.responseText;  
     if (request.status >= 200 && request.status < 400) {  
         // Success!  
         console.log('Success!');  
     } else {  
         // We reached our target server, but it returned an error  
         console.error('Error!');  
     }  
     console.log(JSON.parse(resp));  
 };  

 request.onerror = function (e) {  
     console.log(e);  
 };

Alright, let’s run through some key terminology of the AI service.

Location: This is the subscription location of the service that was selected prior to getting the subscription keys. If you can’t remember the location for some reason, you can go to the Overview screen and find it under Endpoint.
Subscription Key: This is the key that unlocks the service for our plugin use and can be obtained under Keys. There’s two of them, but it doesn’t really matter which one is used.
Image URL: This is the path for the image that’s getting the alt text. Take note that the images that are sent to the API must meet specific requirements:
- Accepted formats: JPEG, PNG, GIF, BMP
- File size must be less than 4MB
- Dimensions should be greater than 50px by 50px

Easy Peasy

Thanks to big companies opening their services and API to developers, it’s now relatively easy for anyone to utilize computer vision. As a simple demonstration, I uploaded the image below to Microsoft Azure’s Computer Vision API.

The service returned the following details:

{  
     'description': {  
         'tags': [  
             'person',  
             'holding',  
             'cellphone',  
             'phone',  
             'hand',  
             'screen',  
             'looking',  
             'camera',  
             'small',  
             'held',  
             'someone',  
             'man',  
             'using',  
             'orange',  
             'display',  
             'blue'  
         ],  
         'captions': [  
             {  
              'text': 'a hand holding a cellphone',  
              'confidence': 0.9583763512737793  
             }  
         ]  
     },  
     'requestId': '31084ce4-94fe-4776-bb31-448d9b83c730',  
     'metadata': {  
         'width': 920,  
         'height': 613,  
         'format': 'Jpeg'  
     }  
 }

From there, you could pick out the alt text that could be potentially used for an image. How you build upon this capability is your business:

You could create a CMS plugin and add it to the content workflow, where the alt text is generated when an image is uploaded and saved in the CMS.
You could write a JavaScript plugin that adds alt text on-the-fly, after an image has been loaded with notably missing alt text.
You could author a browser extension that adds alt text to images on any website when it finds images with it missing.
You could write code that scours your existing database or repo of content for any missing alt text and updates them or opens pull requests for suggested changes.

Take note that these services are not 100% accurate. They do sometimes return a low confidence rating and a description that is not at all aligned with the subject matter. But, these platforms are constantly learning and improving. After all, Rome wasn’t built in a day.

2018: The Year of Artificial Intelligence

Nino Ross Rodriguez — Tue, 01 Jan 2019 19:47:15 +0000

[originally posted at infinite imaginations]

Another year has passed by quickly — this year however, has been consistent. Artificial intelligence (AI) has been constantly making tech headlines and I have been fortunate enough to be able to play around with some of it. As technology grows and evolves at an exponential rate, what does AI have to offer for us in the future?

Artificial Intelligence - AI will continue to improve, for better or worst

JARVIS, Skynet and Hal are a few of the well-known self-aware and sentient computer programs in science fiction. While these characters may not be far away from the distant future, companies like Google and IBM are continuously developing systems to either help us to do a particular task faster or just be better than us humans, hoping that one day robots or androids will be there for us to do the dirty work. AI has at one point has alarmed people of losing their jobs. Just relax because at this point in time, AI is only capable in excelling at very specific areas.

With the aid machine learning, we will see companies utilise AI to automate tasks such as fast and accurate data learning, autonomous mobile robots, digital assistants and conversational platforms to name just a few. We first saw AI been implemented in smartphones such as Apple’s Siri and Google’s Assistant and in the recent months, we have seen a sudden surge of smart speakers being released to the consumers. AI will continue to spread like wildfire across the industry in the years to come. In 2019, we will definitely see more of AI being integrated into other devices, helping us accomplish more tasks quickly without us humans getting our hands dirty.

Conversational Platforms - “Hello, how can I help you today?”

“Ok Google, Hey Siri, Alexa,” the initialising commands of Google’s, Apple’s and Amazon’s personal assistants. The platforms developed by their respective tech companies have paved way and developed speech recognition algorithms with the use of artificial intelligence and machine learning. Chatbots hit the industry and initially posed as a threat for call centres helping to take the load off the people by answering most frequently asked questions or help consumers purchase items through a conversation without the need of a human behind the wheel.

During the peak of its hype cycle, I have experimented with the technology and learned a few things about building a chatbot. One of the most important lessons that I have learned is that “one should never build a chatbot just because of the hype” because this will be the failure of the project. In 2019, consumers are more likely accept these conversational platforms as tech companies continue to develop and improve these platforms. Because it is really easy to create a chatbot, we will see more of these chatbots popping out everywhere We might even be conversing with one without knowing it.

Being online, all the time - Smartphones usage will steadily increase

We all know that our smartphone is our “digital Swiss army knife” and we could not leave home without it. Browsers has been the platform to get users to the internet and tech companies are continuously improving their products to meet the demands of their loyal consumers. Besides improving the battery life, screen size, and camera quality, Google and Apple have integrated AI into their software providing an improved and smoother user experience. Not only that you have a virtual assistant with you in your phone all the time, it also learns and “predicts” the apps that you might be mostly using next at that specific time of the day.

There is no stopping of browsers to improve in 2019 and the years to come. Browsers are gaining features and capabilities that are becoming more “native” and “app-like.” Machine learning has even landed on browsers and I have managed to play around with such technologies that enabled me to detect objects using a camera and a web browser. Developers are even planning to get virtual and augmented reality as a standard to web browsers in able to get more users to have an immersive experience without the need to download an app.

Super Duper Hyper-personalisation - Better customer experience means more loyal customers

I open up my browser and start browsing at Facebook or Amazon and at first I didn’t notice it, but I saw a familiar pattern: I see ads or posts that were related to my past searches. I listen to Spotify, watch videos on YouTube or movies on Netflix and I noticed the music or movies being presented were of my interest. I didn’t tell anyone nor did save any preferences. It just happened. What is this sorcery?

With the advent of personalisation, it wasn’t enough for people to just read their names in emails from sites and services that they have subscribed to. That was already standard in digital marketing. Hyper-personalisation took it a step further by customising the person’s customer experience or journey by providing the things that they want or more interested in. In the years to come, hyper-personalisation with the help of AI will enable companies to provide the things their consumers want or interested at as quick as possible, providing better customer experience and making them loyal to the brand. Better customer experience means more loyal customers. More loyal customers means more money to the company. And I’m a sucker at that.

Autonomous Mobile Robots - Robots running and opening doors, sh*t is getting real

I always thought that drones would only be a rich-man’s toy, flying around in open spaces and take photos or videos that were impossible to be taken by anyone. With drones becoming more cheaper, they are now being utilised to transport packages, food or other goods to places. These unmanned aerial vehicles (UAV) can help deliver medical supplies to inaccessible places. Backed up with AI, these UAVs are not aided anymore by humans. They are programmed in such a way that they know what to do in case something goes wrong. They know where to go and how to come back to headquarters without human intervention.

After watching Black Mirror’s episode titled Metalhead and a short video of Boston Dynamics’ robots, the future got a little bit exciting (or scarier depending on how you look at it). With drones now capable to deliver goods, a robot that can do parkour, and a dog that can open doors, it is certainly clear that the dream of having a robot or android as a helper will be coming to help us in the near future. Maybe one day, we might just have our own Threepio helping us.

Bonus Round: Immersive Experience (AR, & VR) - Merging digital world with physical world

Augmented reality and virtual reality has been going on for some time now. However it only has gain most of its traction in the gaming sector. The most notable AR game is Pokemon Go, where players get to “exercise” by walking around the physical world in search for Pokemons in the digital world. VR games too are attracting popularity by immersing players into another world. They can see, hear and interact in the virtual world that they are thrown into. Tech companies such as Zero Latency and VR Zone Shinjuku offer VR experiences to the public. I was lucky enough to have enjoyed and experienced VR games in Japan and I would say that it is not only fun, but a memorable experience for any gamers.

AR and VR has now expanded their reach beyond the world of gaming. In the next few years, it would reach training, education and tourism sectors just to name a few. Some already has embraced the technology and provided immersive experiences to their customers. Retail companies have invested in AR booths, where their customers can try on clothes without even removing their own in a virtual world. As tech companies like Google and Mozilla developing to make AR and VR to be a standard in browsers, I believe that one day, we will one day see more [immersive experiences in the mobile web](/#/article/vr-and-ar-in-the-mobile-web).

Wrapping it up: The future is bright

It will be an exciting year for 2019 as AI, AR and VR continue to gain traction and technology gets better and cheaper. I see the next year will be the integration of these technological trends into more smart devices and wearables. We could potentially see the rise (or the comeback) of smart glasses. Google Glass wasn’t ready for the consumer market years ago probably because of the price tag, battery life or privacy concerns, but changing the approach and purpose of the smart glass might change the perception of the device.

VR and AR in the Mobile Web

Nino Ross Rodriguez — Mon, 26 Nov 2018 20:03:32 +0000

We have seen people peered into headsets to be immersed into a virtual world, seen people point their devices in all directions trying to look for something interesting around them. VR and AR apps are already available in app stores, and this can be a hindrance to get audiences on board. What if it was easily available in the mobile web?

Virtual Reality

VR is an interactive computer simulation taking the user into a virtual world. The computer generates a simulation and tricks the mind to believe whatever the user is seeing is real. It is usually associated with weird looking goggles, having long wires and mostly for entertainment. It was clunky, had low definition objects and just didn’t know what you need to do in the new virtual world. It was expensive to create, maintain and distribute to the consumer market.

The Sensorama machine (scriptanime.wordpress.com)

With VR catching up with technology at an exponential rate, goggles and headsets are getting smaller and lighter now where people can now wear it with ease. The quality has greatly improved also, projecting more realistic and smoother experiences. Price has also become more cheaper for the consumers. Companies have created their own VR equipment that consumers can buy without hurting their wallets.

Augmented Reality

AR is somewhat similar to VR. Instead of the computer generating the entire environment as a simulation, it creates virtual elements and embeds it in the real world. Just like VR, it required weird looking goggles, wires and were still connected to a computer. A common usage of AR for example is in heads up displays (HUD), where data or information is being displayed in transparent displays without making the users look away from the usual viewpoints.

Virtual Fixtures – first A.R. system (1992, U.S. Air Force, WPAFB)

AR once had a niche market but has now expanded into more broader industries such as gaming, education and businesses. AR is has become more popular with the success of Pokemen Go. As technology gets better and better, tech companies like Google and Apple has invested in incorporating AR in their devices for developers to explore the different possibilities.

Technological Advancement

Though it still requires a computer and other tracking devices, consumers are now able to purchase their own gear to jump into the virtual world. Developers too have been given the opportunity to create their own content and distribute it easily in the market. The most notable names that resonate in the market are Oculus and HTC Vive. Sony soon joined the bandwagon, creating their own VR peripherals which was cheaper than the PC versions.

With these technological advancements, it is not impossible to see that one day VR and AR would be fully supported on our smart devices. Tech giants have already started integrating the technology in smartphones. Companies already have taken the advantage of this by creating rich experiences for their users. While the developers at Mozilla and Google proposing the standardisation of WebXR API, web developers have started to create plugins to make VR and AR work for the web.

VR and AR in the Web

As all developers wait for the WebXR API to be standardised and released to the public, there are a lot of other alternatives where developers can utilise to showcare VR and AR for the web. The most popular one out there is Three.js by Ricardo Cabello (Mr.doob). But it can be daunting for developers who have little or no experience in the WebGL. An easier alternative would be A-Frame, a framework specifically designed to create rich VR experiences without having to know WebGL.

See the Pen Hello World — A-Frame by Nino Ross Rodriguez (@oninross) on CodePen.

A simple VR world like the example above was created only using 17 lines of HTML code. It works well on both desktop and mobile devices and users can dive in immediately without downloading or installing any additional software or hardware. There is an additional icon that is present on the screen where users can choose to be in full screen on desktop and stereoscopic VR on mobile devices. It gets more interesting on mobile devices because the device acts like the camera in the VR world which means wherever the user looks, it's the actual representation in VR.

AR.js demo displaying AR objects using Hiro markers on mobile phones

AR.js was created by Jerome Etienne, focusing on making AR for the web a reality. Though it still relies and requires for markers to display elements on the screen, it's still a relatively big step on bringing AR to the web. The sample above was created only using 30 lines of HTML code (a little bit larger to show 2 markers displaying 2 different objects). Just like it's A-Frame sister, it works well on both desktop and mobile devices.

The Future of VR and AR

Google showcased WebARonARCore at their IO event this year. I managed to play around with the experimental technology in Chrome Canary and I must say that it looks really promising and it's paving a brighter future for VR and AR in the web. Markerless AR on the web would make it more easier for users to get onboard and immersive without the need of downloading an app. The floodgates will surely be wide open when once this feature has been enabled by default and is standardised.

Google Demo of WebXR Device API

Imagine a collaborative virtual world where you only need a smartphone and you don't need to download an app, where you don’t need to buy expensive equipment to experience VR, where you can visualise the information you need anywhere, anytime. It is truly an exciting time for the mobile web.

Are QR Codes Making A Comeback?

Nino Ross Rodriguez — Mon, 30 Jul 2018 11:21:56 +0000

At some point in time, we have seen QR codes plastered all over the place, products, boxes and or posters. It was meant to serve as a bridge from the physical world to the internet, taking users to a webpage, a YouTube video or even make your smart device perform a specific task. The user experience imagined was to help users get to the information quickly. However, it was not the case. Though QR codes are thought to be extinct, these squared dots refuse to die and are rising once again in different forms and use cases.

What ever happened to them?

QR code system was invented to track vehicles during manufacturing and allowing high-speed component scanning. Taking that simple idea and applying it to the web industry, it was supposed to provide users with quick and specific information. If the QR code was on a poster about a concert, the user can simple scan the QR code and take the users to a webpage containing more information about the concert. That is just one of the possible applications.

Though the abbreviation means “Quick Response,” there were pitfalls in the user experience of the technology. At the time of the QR code boom, responsive web design was not yet common and viewing a website on your smartphone was literally viewing the desktop version in a tiny screen. It was a terrible experience. It might be one of the main reasons why users have chosen to abandon and even ignore QR codes when they are in sight.

We barely see QR codes anymore. Even if we did, we wouldn’t be bothered to pick up our smartphones to scan the codes. However, it has risen from the ashes and is being used differently and others have evolved to something else.

“I shall return…” - QR Code

Typing on a small touch screen has always been a problem for most users. Tech companies has always continued to improve keyboard typing on mobile devices ranging from auto-complete to swipe-typing. That partially solves typing for sentences that we normally to in our day to day lives. There are cases that a user will still need to type something out of the ordinary like a very long URL mixed with numbers and characters (need I mention both in small and big letters).

Cue in the QR Code.

WeChat

It is probably one of the mobile apps that has successfully utilised the QR code and there is no sign of stopping. It's the one-stop-app for the Chinese audience where users can use the app to pay bills, order goods and services, transfer money to other users, and pay in stores. Stores would have a QR code on their counters, where the user would then scan these codes. The user would then have to key in how much they will have to pay and that’s it. It has simplified the user journey for the customers and made the experience a seamless one.

Twitter / LinkedIn

Sharing contacts with people on a mobile phone can be tricky most of the time. Especially if you heard the name differently or got a number wrong. Twitter and LinkedIn have made it possible to easily share contacts with each other. Both apps have their own QR code generator and reader, so users would be able to generate and scan these codes in 2 taps. This way the person wouldn’t need to jot down the details and fumble trying to open their smartphones anymore.

Snapcodes / Spotify Codes

One of the factors that confuses users about QR codes is that most of the time, users don’t know what to do with QR codes. The blunt irony was that the QR code reader app was never installed on your device when you needed to scan, or there are no QR codes in sight when you finally have installed it. Snapchat and Spotify have evolved QR code generating into a different level that would reflect more of their brand. With Snapcodes having their “ghostly” logo and Spotify codes with their soundwaves, users would immediately know what app they should be using to scan these codes.

Hidden little gems

With apps like Snapchat, WeChat and LinkedIn paving the way, I believe that QR codes are making a subtle come-back in the industry. The term quick response is now living up to it's name, where both tech giants have finally included the feature in their respective devices. Apple made it seamless by just pointing the camera on the QR code and a preview link to tell the user what it has decoded. Google too included it in their software. Though it required a little bit more steps than that of Apple’s, it still gets the job done.

These little gems have helped improve the user experience for everyone. When cameras from Apple or Android scan these QR codes, it gives a preview link that has been decoded. This way it gives a sense of security to the user that what they are about to enter is something safe. The best part of all, it’s not a third-party app anymore. It would be simply taking out your phone and scanning the QR code to get to the endpoint quickly.

The humble QR code has come a long way from being a disjointed user experience to quietly being reintroduced back into the industry. These black and white in one form or another will aim to help us get from the physical world to the internet. Look out for these things, it may be your window to a virtual world.

Identifying Objects Using Your Browser using TensorFlowJS

Nino Ross Rodriguez — Tue, 12 Jun 2018 08:35:59 +0000

You might be familiar with the TV show Silicon Valley and the “Hot Dog” episode where the cast created an app to simply (and yet hilariously) determine the object as a hot dog or not a hot dog. And it’s not science fiction anymore with applications, like Google Lens, rolled out to most modern smartphones. These days anyone can simply point their camera and get the information they need, quickly.

There are services like Google Cloud Vision API, AWS Rekognition and Clarifai - to name a few - that are available in the market for anyone to implement and use. Though these services let you do to more with less code, it does come with a pay-as-you-go price tag. Also, it's a generic image identifier and may have a different use case.

Enter: TensorFlowJS.

It's a JavaScript library released by the Google Brain Team that brings machine learning for everyone. It was originally written in Python, C++ and CUDA. Thanks to the team, they have ported it to JavaScript, where it is commonly used in the browser. Though TensorFlowJS is not exactly the same as his big brother Python version, the library is already equipped with the necessary APIs to build and train models from scratch, run TensorFlow models and retrain pre-existing models, all the convenience of your browser.

TensorFlow has been circulating around my reading feeds and with Google’s recent IO event, it inspired and pushed me to get my hands dirty in machine learning.

Road To Discovery

Warnings from different sources suggest that, TensorFlow wouldn’t be helpful if you didn’t have any machine learning background. Python keeps popping up as the language of choice for developing in machine learning and it seemed that I needed to learn the basics in order to create to proceed. This was one of many hurdles that I encountered. But I was still determined to create my own image identifier.

Warning: If you don’t have any Machine Learning background, TensorFlow is not for you

The first step is always going back to basics and read the documentations at TensorFlowJS’s website. It seemed to be pretty straightforward at first, but I was wrong. More questions surfaced and I was beginning to believe the warning signs earlier. Maybe I do need to learn about machine learning before I dive into TensorFlowJS. Even scouring YouTube for tutorials and references didn’t help much. I did manage to “create” an image classifier locally in my machine but it was running in Python. I needed it to be client-side just like the Emoji Scavenger Hunt.

Having found the repository to the Emoji Scavenger Hunt and hours of reverse engineering the codes to suit to my needs, I was able to finally create my own image classifier that works smoothly in the client-side.

Teach it as if it was like a 2-year-old

I thought that my biggest hurdle would be developing in Python. I was developing in a Windows environment initially and it was confusing and a pain to set up. But the moment I switched to a Mac environment, everything was smooth sailing. The biggest lesson that I learned was providing the system with good data. My colleagues told me in order to have results with high accuracy, you must provide good initial data.

An analogy to simply understand how machine learning works is, teaching it by showing images as if it was like a 2-year-old, where data is the set of images and the 2-year-old is the machine learning system. For example, if you want the child to learn what an apple is, you would show the child different pictures of apples only. Nothing else should be in the picture. No other fruits, no other elements. After a certain number of pictures that the child has seen, they will be able to know when they see an apple in real-life. On the other hand, if you give the child pictures with apples and oranges, apples and bananas, apples and grapes. The child will get confused when they see those fruits together.

The moral of the analogy is that the images that will be initially fed to the system for machine learning should be easy to comprehend for someone or something who doesn’t know what the subject is.

Riddle Me This PWA

Riddle me this riddle me that
Random riddles taken out from the hat
When you know the answer take a snap
If you are correct I will give you a clap

The goal was to come up with my own image identifier and put it into some good use. The Riddle Me This is a PWA that will show you random riddles of common items that can be found around your home. Your challenge is to find the answer and take a picture of it. If you are correct, you proceed with the other riddles. If you are wrong, well...keep guessing.

Have a go at the link below! Happy hunting!

https://goo.gl/oaVLDu

Through The Looking Glass: An Overview of Visual Recognition

Nino Ross Rodriguez — Mon, 14 May 2018 07:58:05 +0000

Through The Looking Glass: An Overview of Visual Recognition

It’s like something straight out of a Sci-Fi movie — machines, robots and androids being able to identify objects and faces with ease. But these days, early adaptations of visual recognition or image recognition technology have been made available today through services provided by the likes of Google and IBM.

Let’s take a journey of how computers and devices took the first steps looking into our world.

Google Images[1]

Do you remember the days when Google was just a search engine and all you could search for was simple text? Google developers then expanded their search product and gave users results with images. The search engine indexed millions of images and the Image Search was born.

A few years later Google introduced the Search by Image feature, which allowed users to reverse image search directly into Google Search without any third-party add-ons.

“There’s An App For That™”[2]

Technologies were still young and limited back then. IBM Watson was still Deep Blue playing chess. The cloud was still a dodgy place to keep your files. You pretty much relied on Google to search for anything. Image recognition apps or programs were written either by companies with their own algorithm but were too expensive to produce, or piggybacked on Google’s Image Search which was the easier and cheaper solution.

Likely developed at the same time as Google Images, Google went on to release an app version of their image search feature. It was called Google Goggles[3] and it allowed users to search by taking a picture. The app also featured the ability to recognise labels or landmarks without using a text-based search.

This meant people could search for virtually anything, immediately, by using their smartphones and not have the argues wait of getting back to a desktop.

Visual Recognition and Web Apps

Let’s fast forward today’s date, where technology had a sudden growth spurt and gave us, users and developers, endless possibilities to play with.

We now have the ability to tinker with artificial intelligence, machine learning, deep learning, natural language processing and visual recognition to name just a few. Of course, we can always hire developers with specialties in artificial intelligence and machine learning, but it would cost us an arm and a leg. There are services from IBM Watson[4] and Google Cloud Platform[5] that provide developers ease of use, all with an affordable price tag.

More Than Meets The Eye

With the possibilities being endless, here are a few examples on how we can use these technologies at our disposal:

“Eye See It” App

There may be a time our visually challenged friends may need assistance in identifying an object or reading small text. With an app installed on their smartphones, it would be as easy as taking a picture and the app, replying to the user with voice, telling the users what it has seen. A user could take a picture of an unknown object to them and could reply, “I am most definitely looking at a sports car”.

“Plant-Eye-tion” App

A possible idea for the plant lovers out there who may not be as well versed as our gardener friends. Gardeners would be a walking encyclopedia of knowledge of plants, carrying information about the plants they take care of. What if we had an app that by taking a picture of the flower or plant, it would give you the name of the picture taken? Not only that, would it give you the full details of the flower/plant, it could help you keep the thing alive sharing how many times per day it needs to be waters and weather it prefers sun or shade… it can turn the hobbyist into an expert sharing the information relevant to your surroundings.

“Eye Keeper”

Another possibility is curating relevant and safe content to your social media wall in real time. Think about it, you’re at an event and there is a big digital wall that aggregates the pictures taken by the people and its uploaded in social media or in the servers. Currently, you have two choices, you have dedicated team members monitoring and curating the content or you give up full control and allow every post with the relevant tagging to be posted to the wall. One costs a lot of time and money, the other is a huge risk.

The visual recognition service can act as a gatekeeper, analysing the images before it actually reaches the live screen, automatically preventing any inappropriate images being displayed in the big screen.

I Can Show You The World...

Artificial intelligence has taken its first few steps to further see into our world. In the near future, we might not need machine learning and deep learning anymore to “see” and “learn.” Google Lens[6] already took the first step in maximising the potential of the combination of visual recognition, deep learning and augmented reality.

I too, took a step in trying to see what this technology can do. Have a play at our little demo below. I highly suggest that you use your mobile phone for scanning images. Bear in mind though, that this is only a proof of concept. You might still get weird replies from the web app. Happy clicking!

What I Have Learned From Building A Chatbot

Nino Ross Rodriguez — Sun, 06 May 2018 21:37:29 +0000

What I Have Learned From Building A Chatbot

What’s with the hype?

Technology nerds and enthusiasts have always dreamed of having a conversation with an artificial intelligence or AI. The living embodiment of the perfect AI would be JARVIS from the Iron Man movies. No keyboards, no mouse, no stylus. Just your voice, to have a conversation with your virtual personal assistant to do work for you.

But that is science fiction. AI is still in its infancy and it has a long way to go to reach maturity to beat the Turing test.

When Siri came out in the iPhone, it was the first digital personal assistant made for users. I was amazed how it instantly recognizes your requests using voice and comes back with a reply. A few years later, Google Assistant came out not only on smart devices, but in smart speakers as well. It was the first virtual personal assistant that has a two-way conversation. Then there was a sudden boom of chatbots.

What is a chatbot?

A chatbot or a conversational UI is any interface that imitates chatting with a real human. It can be as simple as a chat window in a website or as complex interacting with an AI in a smart device. Whatever the medium may be if there is a two-way conversation, you are interacting with a chatbot.

There are a few types of conversational UIs in the industry, flow type, AI type, and hybrid type.

Flow type is a tree-based kind of interaction, where the user is presented with choices and driven through a specific path. This path is pre-defined by the developer and can only “go” where the interface tells the user to go. An example here would be like the Choose Your Own Adventure books.

AI type relies on artificial intelligence where the user can freely engage and have a real conversation. Something like Google Assistant, Siri and Cortana, there is an AI driving behind all the conversation.

Hybrid type is the most common type of conversational UI and this is where chatbots come in. It’s a combination of flow type and AI where the users are driven through a specific path while they can engage with the chatbot in a conversation.

Do you want to build a chatbot?

As a developer, you would only think that it would be as easy as getting code from the internet and deploy to a server. There was more to it than lines of code and pixels of art.

Lesson 1: “Don’t build one because of the hype”

It was my first mistake and I believe that it’s the golden rule in creating chatbots. Just because its trending now means that one should get into the band wagon and hoping for it to do its job. What I did was basically got Nathan Mk I off the shelf and got my friends to test it. Because it didn’t have a sole purpose, the users who tested it assumed that they were “talking” to JARVIS.

In short, the chatbot should have a purpose. Introduce the chatbot to your audience. Tell them what is his purpose and what can he do. This way you are setting your audiences’ expectations to the level of your chatbot’s capabilities.

Lesson 2: Map the user journey

After concluding that the chatbot’s purpose should be my digital portfolio tour guide, I thought to myself: “how hard can it be? I have an online portfolio, there is my user journey.” I “programmed” Nathan Mk II to capture certain keywords and reply to them accordingly. However, I faced the problem of the possibility of users going to a different section of the site if they wanted to. Connecting them all together was troublesome. I got lost in my mind of how many different permutations can a user go from point A to wherever they wanted to. On top of that, there were a lot of holes that I didn’t foresee. It felt like it never ended.

Lesson learned, mapping the user journey will make a developer’s life easier when it’s time to program the chatbot. You will also foresee any holes that needed plugging.

Lesson 3: Build the script

Since I didn’t map out the user journey properly, I thought I could write the script on the fly. Again, “how hard can it be? I have my website, there’s my script.” Copy and paste should do it. As I progressed building my chatbot, I found it hard to come up with good replies and answers. It was either to broad, boring or had open-ended questions. This strayed the users from telling the chatbot the right keyword. ‘

You need to lead your users. Guide them on what actions can be used to progress further. If you can’t come up with a meaning script, hire a copy writer. Solving logic is one thing and having a meaning conversation is another.

Lesson 4: Add some flavor to it

Chatbots are still an inanimate object and text conversation can get boring quickly. However, it will say to whatever you program it to say. You can always mix it up with animated GIFs, photos, emojis, etc. (if applicable) just to keep the conversation interesting and fun. Give it a little personality as well. Giving the chatbot a little personality will make the users remember it.

Lesson 5: “I need a human”

After endless hours of using and testing, I got used to navigating around the site using Nathan. I deployed it and let my friends try it. In the end, users got stuck in an infinite loop of “I don’t know” replies. That could mean bad experience for the user. Instead of trapping your users in limbo, make sure that if your user is stuck at most 3 “I don’t know,” give the option to talk to a human. We need to make our users happy by reaching their objective.

Nathan Mk III

It was a good learning experience in creating Nathan and I had a lot of fun exploring this emerging technology. I learned that planning is essential because this is the backbone of the entire chatbot project. Just like any project that you may encounter, we need plan for the project, foresee the issues that may arise, and ask the questions needed. If planning is done properly, it would be smooth sailing from there onwards.

Chatbots are popping everywhere and uses natural language to communicate with their users. We can further enhance the user experience by utilizing voice to communicate. Voice interactions and voice user interface will be the next big thing in the industry. Here is a demo of what came out of my learning experience. Have a play!