DEV Community: Zachnology

Do Your Loops Have Knots?

Zachnology — Wed, 15 May 2024 16:22:06 +0000

Avoiding I/O performance pitfalls

Cover Photo by Önder Örtel on Unsplash

I/O operations are expensive. As an application scales I/O operations can become serious bottlenecks. Whether an application pulls data from a database, a web service, or an IoT device, it can suffer from the same problem: expensive I/O operations inside of loops. Here we will view the problem through the lens of Entity Framework in C#.

Identifying the Problem

Without experience with this issue, it can be easy to overlook during initial development. Unit tests, integration tests, and manual QA testing may not use large enough datasets to identify problems preemptively. If not caught ahead of time, the issue will certainly be noticed in production. Here is what it might look like:

public User[] GetUsers() {
    User[] users = GetRawUsers();

    foreach (var user in users)
    {
        // ...
        SomeExpensiveOperation(user);
        // ...
    {

    return users;
}

Let’s say that GetRawUsers() and SomeExpensiveOption() take 100ms each to run. If there are only 10 users in the users array, then GetUser() takes 1100ms to run.

100ms + (100ms * user_count)

You can see the problem with scale now. The cost in time increases linearly with the number of users. With only 100 users, it will take 10,100ms to run the GetUsers() method and even worse if SomeExpensiveOperation() contains a loop of its own.

How to address it

In real-world scenarios, the problem code might be more subtle than the above contrived scenario. Here are some ways to identify such cases.

Be mindful of when a query is executed

One common cause for this issue when using Entity Framework, and I suppose other frameworks that provide query building, is not paying attention to when the query is executed. For example:

var usersWithGreenEyes = db.Users.Where(u => u.EyeColor == "green");
var ages = List<int>();

foreach (var user in usersWithGreenEyes)
{
    ages.Add(DateTime.Today.Year - user.DateOfBirth.Year);
}

Because Where() does not execute a query, each iteration of the loop will fetch the in-scope user record from the database creating a database call per user.

In Entity Framework, this can be solved by ensuring the query is executed before operations on the data begin by calling a method that takes the IQueryable returned by Where() and produces an List or Array.

 var usersWithGreenEyes = db.Users.Where(u => u.EyeColor == "green").ToList();

Now only one query will be executed and before the loop starts.

Be mindful of lazy loading

In Entity Framework, if lazy loading is turned on, seemingly benign references to child entities and child collections, could execute a query. For example:

var users = db.Users.ToList();
var distinctTraveledCountries = new HashSet<string>();

foreach (var user in users)
{
    distinctTraveledCountries.UnionWith(user.TraveledCounties);
}

In the above example, we made sure the execute the query before entering the loop, but if lazy loading is turned on, each access to a child collection (like user.TraveledCountries) will still execute a database query to get the child data.

This can also happen when using a mapper library. For example, if I have an array of user entities and I want to map that to an array of some type of user view that contains the same child collection of traveled countries.

IEnumerable<UserView> userViews = mapper.Map(userEntities);

The same lazy loading mechanism from the previous example will cause issues here as well since the mapper library will internally access each user in the array.

Often the solution for both of these cases is the same: eager-load the child collections and child entities, either explicitly or by mapping as part of the query.

// Eager loading explicitly
var usersWithChildren = db.Users
    .Include(u => u.TraveledCountries)
    .Include(u => u.AccountInfo)
    .ToList();

The above will eager load both the TraveledCountries collection as well as the child member AccountInfo, so that those properties will be prefetched along with the rest of the User entity.

In Entity Framework this generates a single query which will include left joins on the child data. In some cases this could cause performance issues itself. One solution would be to use a split query. This looks almost the same but underneath it executes a query for the parent and a query for each included child. Instead of left joins, the child members are joined with an inner join.

// Eager loading explicitly as split query
var usersWithChildren = db.Users
    .Include(u => u.TraveledCountries)
    .Include(u => u.AccountInfo)
    .AsSplitQuery()
    .ToList();

The below will map any needed fields to a custom object type by only selecting the fields needed.

// Mapping in query
var userViews = db.Users
    .Select(u => new UserView {
          Id = u.Id,
          Name = u.Name,
          AccountInfo = u.AccountInfo
        })
    .ToList();

Use Paged Data and Work in Small Batches

You may find that while eager-loading data fixed your performance issue as far as the loop is concerned but it caused another performance issue with a query result set that is too large or a query that takes too long to run. In this case, it may be helpful to limit the scope of an operation. You can do this by implementing pagination. If the application is a web service, you can limit the amount of data returned and provide metadata for fetching the other pages. A response could look something like this:

{
  "metadata" {
    "page": 3,
    "pageCount": 10,
    "limit": 20,
    "links": {
      "self": "https://myapi.com/users?page=3&limit=20",
      "first": "https://myapi.com/users?page=1&limit=20",
      "previous": "https://myapi.com/users?page=2&limit=20",
      "next": "https://myapi.com/users?page=4&limit=20",
      "last": "https://myapi.com/users?page=10&limit=20",
    }
  },
  "users": [
    //...
  ]
}

If the application is an automated process, it can work in small batches instead of the entire dataset at once. Pagination for query execution in Entity Framework looks like this:

public IList<User> GetPagedUsers(int currPage, int pageSize)
{
    return db.Users
      .OrderBy(b => b.UserId)
      .Skip(currPage * pageSize)
      .Take(pageSize)
      .ToList();
}

Identifying Before Production

The symptoms of the issue we are discussing usually don’t reveal themselves until applied at scale, so it can be easy for it to get past traditional automated testing and manual QA. Here are some ways to help identify it:

Developer education. Learning about pitfalls so they can be addressed during development.
Setting performance requirements associated with features. Whether or not you have an SLA you must adhere do, you can still include a threshold in the feature requirements before development begins.
Code reviews
Realistic testing against production-like environments (the earlier in the release process the better).

Need help with your software project?

My day job is working as a software engineer for Trendsic where I help businesses large and small plan and develop secure and scalable software. Reach out to me at contact@zachnology.io to discuss how I can help bring your ideas to life.

Making Websites for Spies

Zachnology — Tue, 07 May 2024 04:01:37 +0000

Abstract JavaScript Events

Cover photo by Agence Olloweb on Unsplash

Have you ever thought about how the CIA communicates with informants all over the globe? There’s the clandestine meeting. Maybe slipping of a note into a pocket as someone walk by. A removable brick in a wall. But is there a way to communicate in plain sight?

It is believed that during the mid 2000s to at least the early 2010s the CIA created a series of fake websites that handlers used to communicate with informants. To the unknowing eavesdropper, it would look like an informant was just visiting a local news website, but the ordinary-looking search box was actually a password input field that would give them access to the communications functionality.

Before I ever heard about this practice, I had built a proof of concept for clandestine interaction on websites too. And since I didn’t use it for international espionage, what I’m about to share is completely declassified…

Abstract Events

We can detect and listen to pretty much any input performed on a website. Whether we need to simply detect a focus or blur event on an input element or we want to track mouse movement across the page, we have access to the data and the triggering event.

This is all fine and dandy, but what if I want to implement keyboard shortcuts for my website? Two of the more popular use cases for this is making enter do something other than submitting a form and creating desktop-like shortcuts for web apps (like Ctrl+ shortcuts).

The case of overriding the behavior of the enter press is mostly trivial. You’ll need to listen to the keydown or keyup event, then check if enter is the key that has been pressed. If so, cancel the default behavior and then do whatever you want enter to do.

document.addEventListener("keydown", (event) => {
  if (event.key === 'Enter') {
    event.preventDefault();
    // your custom logic here
    return false;
  }
};

It works the same way for the Ctrl+ shortcuts but you just check if ctrl is also pressed.

document.addEventListener("keydown", (event) => {
  if (event.ctrlKey && event.key === 'z') {
    event.preventDefault();
    // your custom logic here
    return false;
  }
});

These simple key “overrides” got me thinking about making an event listener factory that took abstract instructions and produced a JS event handler. My goal was to come up with some unique input events and provide an easy to use interface for it.

What if I wanted to listen for more complex input that needed more information than a single mouse or key event could give me? Can I trigger special behavior if a key word is typed in even if an input element is not in focus? Can I make a button behave differently depending on how many times I clicked it, or how fast I clicked it? If we buffer the input, we can easily track these types of events.

I created a proof of concept npm package called abstract-events that exposes a simple interface for this type of event. The create() method accepts 2 parameters: options and a callback. Here are some examples:

Count Events

The first type of abstract event is a count event. It triggers once an input event has been triggered a defined number of times.

Click Count

The code below attaches an event handler to a buttons onclick event that triggers when you’ve clicked the button 4 times.

const abstractEvents = require('abstract-events');

let someButton = document.getElementById('someButton');
someButton.onclick = abstractEvents.create(
    {
        type: 'count',
        count: 4
    },
    () => alert('You clicked the button 4 times!')
);

Timed Click Count

Here’s the same example but ignores clicks that happen outside a 1.5 second window. This could be used to ensure the clicks were intentional to trigger this event.

const abstractEvents = require('abstract-events');

let someButton = document.getElementById('someButton');
someButton.onclick = abstractEvents.create(
    {
        type: 'count',
        count: 4,
        timeout: 1500
    },
    () => alert('You clicked the button 4 times within 1.5 seconds!')
);

Parsed String Events

You can also trigger phrase events. This listens for a specific string before triggering.

Specific String

Here the event is attached to the document’s keydown event. It triggers once you’ve typed in the string “secret”, regardless of focus.

document.addEventListener('keydown', abstractEvents.create({
    type: 'phrase',
    phrase: 'secret'
}, () => alert('You did it!')));

Timed String

Just like click example, you can limit the phrase event to a time window. In this example, the event only triggers if the string “faster” is typed within 1.5 seconds.

document.addEventListener('keydown', abstractEvents.create({
    type: 'phrase',
    phrase: 'faster',
    timeout: 1500
}, () => alert('You did it fast!')));

You can play around with these events on the live demo page.
The source code for the library is available on github.

A Note on Security

Don’t use this for actual privileged access. Because these events are defined entirely in the frontend, they are not suitable for authentication. Someone could easily just look at the source of your web app and see how to bypass the security. Further, having a single global passphrase makes any system easier to hack. Deploying something like this at scale may also make your sites easily discoverable just with google dorking, which brings us back to the CIA’s implementation…

A “Security Through Obscurity” Failure

The CIA’s spy website project did not end well. The sites were just a thin façade for their real purpose. And since websites are constantly up, they can be submitted to scrutiny for longer periods of time than more traditional CIA informant encounters. If you were suspicious of a website someone was viewing, you just had to look at the page source and see that the search text box was actually a password field. Research by Citizen Lab found 885 such websites just by using OSINT (Open-source Intelligence) and discovered evidence that the poor design of security features for these sites led to the compromised identity and arrest of informants worldwide.

Conclusion

This article ended up serving two purposes. One was hopefully food for thought in thinking outside the box on how to use JS events for web input. The other is the importance of security for our web applications, not just using proper authentication or protecting sensitive data but also that we should protect the means of security so attackers don’t have an easy starting place.

Need help with your software project?

Automation: Holidays

Zachnology — Tue, 07 May 2024 03:54:19 +0000

Procedurally calculated holiday dates in JS

Cover photo by Towfiqu barbhuiya on Unsplash

Have you worked on a project that required a periodically updated list of holidays? How did you find those dates?

I’m seeing a pattern

I’ve seen a lot of databases and one table that consistently pops up in corporate databases is the “holiday” table. HR will store paid company holidays in it, or marketing will use it to schedule email blasts, or it might be used to automatically populate the calendar on the company’s public website.

What is interesting about this database table is that it is usually manually populated periodically…usually when something breaks and IT realizes it’s time to update the holiday list again. And that may be totally find for your project.

The plot thickens

A few years ago I was working on a project that was using one such holiday table for marketing purposes. I decided to see what other solutions were out there. Googling the matter, I came across several PAID services offering a holiday API. You paid to access an api that would return dates of holidays. It seemed that even companies focused on solving this problem were just using their own manually updated holiday list and charging you to access it with the promise of keeping it up to date.

Is there a better way?

Do you really need an API to provide easily consumable holiday dates? As it turns out you can procedurally determine the date of any holiday. While some holidays always fall on the same date, most holidays are determined by some algorithm.

Federal Holidays

Federal holidays in the United States are created by law, meaning there are defined procedures to determine the date of a given holiday. What’s more, there are procedures defined to determine what day a holiday should be observed should the holiday fall on a weekend (so employees can still enjoy a day off).

For example:

Memorial Day is the last Monday of May
Thanksgiving Day is the last Thursday of November
Christmas Day is always the 25th of December but the federally observed day is the nearest weekday

Religious Holidays

Similar to federal holidays, there are rules defined by religions for when each religious holiday should be observed. Some branches of a religion might celebrate the same holiday on a different date governed by a different procedure.

Other Holidays

Some observed holidays are not defined by the government or religions but there at least exists some rule so everyone knows when to observe it.

Easter

Most holidays are easy enough to calculate. This one, though, might be the cause of so many just giving up and manually punching in the date for holidays each year. The good news, is we only have to write the logic to determine it once.

Easter has several methods of calculation depending on where you are in the world. I’ll describe the calculation used in the United States. It’s calculated using the “Metonic Cycle” with logic determined by the Easter computus.

The gist is this. There is an association between a “Golden number” and a “Paschal full moon date”. (Technically the golden number is associated with a number called Epact which determines the full moon date, but all these numbers have a 1 to 1 association so we can just jump from golden number to the full moon date.)

The association is as follows:

Golden Number  Paschal Full Moon Date
1..............14 April
2...............3 April
3..............23 March
4..............11 April
5..............31 March
6..............18 April
7...............8 April
8..............28 March
9..............16 April
10..............5 April
11.............25 March
12.............13 April
13..............2 April
14.............22 March
15.............10 April
16.............30 March
17.............17 April
18..............7 April
19.............27 March

First you need to determine the Golden number. This is done by taking the modulus of the year divided by 19, then adding 1. The calculation for 2024 is as follows:
2024 mod 19 + 1 = 11

11 is our Golden Number. We do a lookup on our computus table and see that 11 is associated with the Paschal full moon date of March 25.

The rule then says that Easter should be observed the next Sunday following this date. If the Paschal full moon date lands on a Sunday, Easter will be observed the following Sunday.

The closest Sunday following March 25, 2024 is March 31, 2024 so that is our date for Easter 2024. Now Google it to check if I’m right.

Now we can determine the date of other holidays dependent on Easter:

Mardi Gras: 47 days before Easter
Ash Wednesday: 46 days before Easter
Palm Sunday: 7 days before Easter
Good Friday: 2 days before Easter

Enter Holidate

After learning all I did about holiday calculation, especially that of Easter and its related holidays, I decided to publish those algorithms for anyone to use without going through the mess of writing the logic themselves.

You can install the dependency free npm package with

npm i holidate

or you can check out the source code on github.

It has basic features allowing you to query holidays by year, locale, language, and holiday type.

Most of the existing holiday logic is for US-based holidays and available in English and Spanish. If you’d like to help add holidays from other countries or from cultures or religions not currently represented in the holidate library, feel free to open a pull request and contribute to the repo.

Now go forth and automate your holiday lists!

Apply It Elsewhere

This article is about holiday calculation but the idea and process could apply to any date calculation regardless of being a holiday or not. The holidate library is a very basic rule engine. It relies on holidays being defined by rules. The code processes these rules to determine the date of a holiday for a given year. This same process could easily be applied to any type of recurring event.

Need help with your software project?

My day job is working as a software engineer for Trendsic where I help businesses large and small plan and develop software ideas using lean practices. Reach out to me at contact@zachnology.io to discuss how I can help bring your ideas to life.

The Infrastructure Iceberg

Zachnology — Tue, 07 May 2024 03:42:53 +0000

I just need a simple app on the app store, that's all...

Cover photo by Alexander Hafemann on Unsplash

Let’s say you are seeking a consultant to build a software product for you. You might be inclined to hire your friend’s buddy who builds apps on the side since he can do it cheap or at a low hourly rate. All you need is the dev to build the app and then it’s self sufficient, right? I mean the app store hosts the thing after all!

SaaS products are like an iceberg. The end user sees a pretty mobile app icon or a nice web site but rarely is all the functionality self-contained. Often there are many layers of technology behind the scenes that support that app, not to mention infrastructure and security maintenance. When factoring in the viability and cost of implementing an idea, you should consider all it takes to support an app and what maintenance for it looks like.

To help explain different infrastructure components, I’ll use the example of a mobile app that helps you manage your pantry. You can create an account, store the contents of your pantry, get recipe recommendations based on what you have in stock, purchase food to replace what’s out of stock, and receive alerts when certain items are on sale. What does it take to support such an app?

App Store Developer Account

In order to publish an app to the Apple App Store or the Google Play Store, you will need a developer account on the corresponding store’s platform. Although the name includes “developer” you can think of these accounts as vendor accounts that allow you to publish mobile applications under your company’s name. You can give your developer team limited access to your account via specific permissions to manage the app they are developing for you.

Authentication Provider

Because our example app allows users to create user accounts and store information tied to their user account, you will need a tool that manages user accounts and provides authentication functionality. While this can be a home-grown solution, it’s usually more secure and more cost effective to use a third party provider. Such third party providers include Auth0, AWS Cognito, and Microsoft Entra ID.

Email Service

When there are user accounts, there are emails. At the very least, you’ll need to email users as part of setting up an account or for password recovery. You may also choose to send marketing emails related to your app or to send notification alerts. Regardless of the reason, you’ll need a way to send automated emails. In order to do so you’ll either need your own SMTP server or use a third party email service to handle the sending. Some examples of third party email services are SendGrid and Mailchimp.

Data Store

Because the app allows users to recall the items in their pantry, you’ll need a place to store that information across login sessions and devices. Some form of database or datastore will be needed. These can be a managed database service you pay for through a cloud provider like Amazon Web Services, Google Cloud, Microsoft Azure or it can be a database server installed on a bare-metal our cloud server you manage.

Payment Processing

Since the app allows users to purchase items through the in-app store, you’ll need a way to process payments for those orders. This means a 3rd party integration with a payment gateway. These include Stripe or Paypal, or payment gateway aggregators like Chargebee or even Shopify.

Servers

For your app to communicate with all of the above services and provide business logic for your app, you’ll need some form of a backend API or web service. This a service running on a server somewhere that communicates with your app to interact with the database, validate users, send emails and facilitate payment processing. Depending on the architecture of the backend, processing power required, bandwidth, and availability zone requirements, this could be multiple servers across multiple time zones and countries.

Since our example app sends alerts to users, we’ll need to a service to send push notifications to them.

Web Presence

You may also want your app’s functionality to be available on the web. This involves developing a web application viewable on desktop and mobile that provides the same functionality and will also need hosting. Even if your app is mobile-download-only, you may still want a landing page website for those using search engines to find your app. There are many hosting providers for web. For a simple landing page, pretty much any static site host will do. For a web application you may need a more custom hosting configuration and infrastructure depending on the technology used.

UI/UX

You’ll want to collaborate with a UX designer to create the look and feel for your app and also how the users interact with it in an intuitive way. If you have a landing page or web version of your app, you’ll want to have those designed with a matching look and feel. If you are sending emails, you will want the email design to match the app.

In our example app, users are sent alerts for sales they may be interested in. This may include advertisements from third parties. We’ll need to either coordinate with an ad service or collaborate with specific vendors to serve up their ads in our app.

Mobile Platforms

While mobile apps on both Android and iOS may look the same, the technology and process used to build them natively are drastically different. Some apps can be created using a cross-platform framework that allows code to be written once and compiled for each mobile platform (more or less). Some apps because of performance, sensor accessibility, or other reasons may need to be developed independently for each platform.

Security

Because your app stores information about its users and payment information, you must do your due diligence in securing that data. That includes securing the database and network used by the platform. It also includes using proper encryption for appropriate data, applying security patches as they become available, and responding to security threats to keep your platform secure.

Ongoing Support

So far we’ve discussed several components that may be part of a software product but are not known to the end consumer. One of the difficulties for small businesses and solo entrepreneurs is the management of all of these components to support their product. While your friend’s buddy may be able to set up some of these components, who is going to manage them while you run your business or as your business scales up? As you move from proof-of-concept to serving real customers, you’ll want to partner with a company that can provide that maintenance and ongoing support.

Conclusion

There is a lot more to running and maintaining successful software platforms that meets they eye. Hopefully this information can help you get a better idea of cost of investment and give you real talking points to discuss when shopping around your project.

Need help with your software product?

My day job is working as a software engineer for Trendsic where I help businesses large and small architect scalable and sustainable software platforms. Reach out to me at contact@zachnology.io to discuss how I can help you with your next project.

Idea Validation for the New Tech Entrepreneur

Zachnology — Mon, 06 May 2024 18:31:23 +0000

So you've got an idea for an app...

Cover Photo by Rishabh Dharmani on Unsplash

Not long ago, I found a post in a tech-focused entrepreneurship forum that went something like this:

My business partner and I have been working on an engineering software tool since we were in grad school. After 10 years of work, it is now full-featured and ready for the market. However, we are having trouble selling it. We have had lots of meetings but no one is signing contracts. Does anyone have advice?

The comments all reflected deep sadness that this team of developer-entrepreneurs had wasted so much time and money only to find out that no one wanted the product they had made. Ten years is a long time but their strategy is actually a fairly common one, especially for technical entrepreneurs.

As developers and engineers, we are hypersensitive to inefficiencies and sometimes to a fault. Instead of asking ourselves “is someone willing to pay for this?”, we just get started building our ideas right away with the hope that if our product has enough fancy features someone will buy it and surely the price they pay and the volume of sales will compensate use appropriately for all of our hard work.

This article aims to help address this behavior of putting the cart before the horse by encouraging the validation of ideas before their implementation and encouraging experimentation as a means of validating ideas. This means putting your idea to the test to see if it is actually a viable product that can make real money.

The goal of idea validation is to prove that someone is willing to pay for your product and that there is a big enough audience of such people to make a profit.

What Constitutes a Validated Idea?

It is important to be honest with yourself and to disregard false-positives. For example:

Someone only telling you that your idea sounds great is not validation.
Your mother wanting to invest is also not validation.
Even someone using and liking a free prototype is not validation.

The goal of idea validation is to prove that someone is willing to pay for your product and that there is a big enough audience of such people to make a profit.

Disclaimer

I’m not a marketing expert. My expertise is in implementing dreamed-up software ideas and that is the point of view from which this article is written. I understand that creative marketing and clever business strategy can overcome some of the obstacles mentioned here but I feel the advice here is still relevant for the majority of startups and entrepreneurs.

Self Assessment

Here are some questions that it would be good to answer about your product before significant investment. I ask these types of questions to clients in our first meeting.

1. What Problem does my idea solve?

1a. Am I trying to solve a real pain point or is it just a mild inconvenience?

Real Pain Point:

My child’s school bus arrives at inconsistent times. This worries me because I don’t know if my child is missing or if the bus is just late. I wish there were a notification system to alert me to the status of the bus to give me piece of mind.

Just a mild inconvenience:

All my favorite apps have dark mode except MS Word. I want to create a word processer that has native dark mode.

2. How unique is my idea?

2a. Is there already a product on the market that solves the same problem?

I want a website where people can go to talk about motorcycles. I want to be able to create a post about motorcycles and people can comment on it and communicate with each other.

Why should customers use your platform instead of just making a Facebook group for motorcyclists or joining one of the many such Facebook groups already out there. If you intend to compete with the social media giants you’ll need to cater to the motorcycle community in a way they do not or cannot.

2b. What differentiates my product from others and incentivizes potential customers to choose my solution over what already exists?

Little Differentiation:

My idea is a calculator app that lets you change the background color.

More Differentiation:

My idea is a calculator app that has shortcuts for common equations used by surveyors doing field work. Instead of fiddling around with a regular calculator and punching in lots of numbers, they can quickly and easily get the data they need and get back to work.

2c. Could a competitor easily make an update to their existing product to solve the same problem?

MS Word doesn’t have dark mode so everyone that likes dark mode will want to use my word processor.

What happens if Microsoft decides to add dark mode tomorrow? Does that invalidate my entire business?

3. Who is my target audience?

3a. Is it a small, large, niche or broad audience?

Too Small an Audience:

My idea is an app for people with gen 1 android phones that allows them to blah blah blah.

Too Broad an Audience:

I want to make an app that does everything for everyone and we’ll market it to all humans.

Well Defined Niche:

My idea is a web app that focuses on helping the desert landscaping industry source climate compatible plants and products.

3b. Is the audience accustomed to paying for such products?

Studies have shown that iOS users spend more on apps and on in-app purchases than their Android counterparts. How do I account for this with my idea?

3c. Would this audience pay the price I want/need to charge?

My idea may provide a “better” experience than its competitors. Does this add to infrastructure or licensing costs that will put my product’s price out of range of what customers are willing to pay?

4. Does my idea solve a problem for a particular industry?

4a. Are there regulatory factors to consider for a product in this space?

There may be regulatory or legal factors based on the industry that could increase cost of development, legal matters, or require that I hire a security or compliance officer. I need to be aware of these cost to plan accordingly.

4b. What is the community like in this industry?

The community in this industry may be averse to change or use of certain technology. An industry that relies on precision of information and data may not welcome tools that use AI. Does my product threaten the jobs of members of this community? How does that affect my ability to market to this community?

4c. How do other successful companies market to the industry?

Can I take advantage of the findings of other companies to know how best to reach this community?

5. What are the legal considerations to implementing and selling my idea?

5a. In which locales can this product be sold and used?

Can I sell this product in all 50 states? Does California require additional legal considerations? Can I sell this product in other countries?

5b. Will I need to collect or store data that is sensitive in nature?

Depending on the industry my product serves, I could be legally compelled to comply with certain laws or standards to protect the data I store on behalf of users. For example if my product serves the medical industry, it will probably need to be HIPAA compliant. I will need to factor that into my budget for development, infrastructure, and maintenance.

6. On what types of devices will my product run? Am I targeting mobile, web, desktop, other?

Where do my customers expect to interact with products like mine? Will they be in an office where it will be more convenient to use on the desktop with a keyboard and a large monitor? Is lots of typing required? Do I need to design the user experience to require minimal typing because users will be on mobile devices?

And more…

I don’t mean to give an exhaustive list of self-assessment questions but rather an idea of the types of questions you should be thinking about. You want to identify potential problems that could hurt or inhibit viability. You may need to revisit some of these questions as you iterate upon your idea.

Lean on the wins and learn from the failures.

A Scientific Approach

What if you learn that part of my idea is not viable? That’s ok! Not only is it expected, it should be considered a win. You learned this valuable information now instead spending thousands of dollars and months of time working on something that would no one would buy. A core tenant of lean methodology is to fail fast. Find out what won’t work quickly so you know to go in another direction before wasting resources. Now you can pivot and adjust your plan to compensate. Lean on the wins and learn from the failures.

By completing an initial assessment of your idea, you’ve completed an experiment. You had an idea, put it through tests, and hopefully you have learned something. At this point you should have identified items that need to be changed to your product to make it more viable. This process of experimentation can be repeated throughout the life of your product, from idea to prototype to MVP to initial release to feature updates. The concept is to make informed decisions about the direction of your product based on real evidence.

Conclusion

What could our friends from the forum post have done differently and what could have been the outcome? Were they wrong to be passionate about an idea? No. What they lacked was proof that their product filled a real need. Now imagine what they could have accomplished in those 10 years if they had tested the viability of their product first, experimented and failed early, and iterated on their idea making changes based on real evidence.

Up Next…

Stay tuned for the next article, “The Infrastructure Iceberg” where we will build a hypothetical app and learn about the costs of supporting a fully-loaded SaaS product

DEV Community: Zachnology

Do Your Loops Have Knots?

Identifying the Problem

How to address it

Be mindful of when a query is executed

Be mindful of lazy loading

Use Paged Data and Work in Small Batches

Identifying Before Production

Need help with your software project?

Making Websites for Spies

Abstract JavaScript Events

Abstract Events

Count Events

Click Count

Timed Click Count

Parsed String Events

Specific String

Timed String

A Note on Security

A “Security Through Obscurity” Failure

Conclusion

Need help with your software project?

Automation: Holidays

Procedurally calculated holiday dates in JS

I’m seeing a pattern

The plot thickens

Is there a better way?

Federal Holidays

Religious Holidays

Other Holidays

Easter

Enter Holidate

Apply It Elsewhere

Need help with your software project?

The Infrastructure Iceberg

I just need a simple app on the app store, that's all...

App Store Developer Account

Authentication Provider

Email Service

Data Store

Payment Processing

Servers

Web Presence

UI/UX

Ads

Mobile Platforms

Security

Ongoing Support

Conclusion

Need help with your software product?

Idea Validation for the New Tech Entrepreneur

So you've got an idea for an app...

What Constitutes a Validated Idea?

Disclaimer

Self Assessment

1. What Problem does my idea solve?

1a. Am I trying to solve a real pain point or is it just a mild inconvenience?

Real Pain Point:

Just a mild inconvenience:

2. How unique is my idea?

2a. Is there already a product on the market that solves the same problem?

2b. What differentiates my product from others and incentivizes potential customers to choose my solution over what already exists?

Little Differentiation:

More Differentiation:

2c. Could a competitor easily make an update to their existing product to solve the same problem?

3. Who is my target audience?

3a. Is it a small, large, niche or broad audience?

Too Small an Audience:

Too Broad an Audience:

Well Defined Niche:

3b. Is the audience accustomed to paying for such products?

3c. Would this audience pay the price I want/need to charge?

4. Does my idea solve a problem for a particular industry?

4a. Are there regulatory factors to consider for a product in this space?

4b. What is the community like in this industry?

4c. How do other successful companies market to the industry?

5. What are the legal considerations to implementing and selling my idea?

5a. In which locales can this product be sold and used?

5b. Will I need to collect or store data that is sensitive in nature?

6. On what types of devices will my product run? Am I targeting mobile, web, desktop, other?