DEV Community: Henri Idrovo

What is the cheapest deploy option on Digital Ocean?

Henri Idrovo — Thu, 22 Dec 2022 17:10:10 +0000

I'm still new to the web stack. One of the first cloud providers I deployed web apps to was Heroku, and that's because that's what the developer behind Bernie Sits used to deploy his viral website back in early 2021. Also it was FREE!

But since then, Heroku has unfortunately decided to stop offering a free tier, and that sucks.

Can't cry for too long. So where do I deploy to now? Enter Digital Ocean. So whats the cheapest option I have on Digital Ocean? See below for what I found. 512 MB ram, single core cpu, $5 per month.

That sounds pretty good right? Well yes, until I found what kind of server Github is giving to developers for their new Codespaces feature for free. 4GB ram, dual core, 32GB disk space. See below.

Hmmmmm. That really put things in perspective 🤔

Design a Web Crawler

Henri Idrovo — Wed, 16 Jun 2021 15:33:55 +0000

What is a web crawler?

Search engines like Google need to find and index new web pages frequently so that users can quickly fetch the latest information. Record labels need to monitor for cases of copyright and trademark infringements. These are just some of the use cases for a web crawler.

The Scenario

Let's consider a use case where we will need to process 1 billion web pages per month and store the pages content for up to 5 years.

Estimate the Storage Requirements

The number of queries for web pages per second (QPS):

1 x 10⁹ pages / 30 days / 24 hours / 3600 seconds = 400 QPS

There can be several reasons why the QPS can be above this estimate. So we calculate a peak QPS:

Peak QPS = 2 * QPS = 800 QPS

The average web page size we will use is:

500 kb

The storage required per month for storing pages:

1 x 10⁹ pages X 500 kb = 500 tb

The storage required to store pages for 5 years:

500 tb X 12 months X 5 years = 30 pb (petabytes)

High Level Design

Below is what the logical flow will look like at a high level. We will then cover each step in some detail.

Seed URLS

The seed urls are a simple text file with the URLs that will serve as the starting point of the entire crawl process. The web crawler will visit all pages that are on the same domain. For example if you were to supply www.homedepot.com as a seed url, you'l find that the web crawler will search through all the store's departments like www.homedepot.com/gardening and www.homedepot.com/lighting and so on.

The seed URLs chosen depend on your use case. If you were a record label creating a crawler to find all instances of copyright infringement, you might use seed URLs were you are most likely to find copyright infringements like www.soundcloud.com and www.youtube.com.

URL Frontier

You can think of this step as a first-in-first-out(FIFO) queue of URLs to be visited. Only URLs never visited will find their way onto this queue. Up next we'll cover two important considerations for the URL frontier.

Politeness: Our web crawler needs to avoid sending too many consecutive or parallel requests to the same server, or else you have the possibility of causing a denial-of-service(DOS) attack. In order to avoid flooding the host server with requests, we can implement a strategy where all requests to a particular host are processed by only one process at a time. This avoids parallel requests to the same server.

The above design shows the components that manage politeness. The mapping table stores a mapping between host domain (e.g. www.homedepot.com) to a single and specific queue. The queue router uses that mapping to send URLs to the appropriate queue. Each queue (1,2....N) then contains URLs from the same host. Each download process is mapped to queue and receives one URL at a time. A delay between each download task can also be added.

Priority: Not all web pages have the same level of importance. A new post on a reddit group has less priority than a new post on the associated press, for example. So we need a component in our workflow to manage priority. We'll use the same design that we used for the politeness component, except in this case the different queues represent different priority levels. And the queue selector is responsible for selecting URLs from the high priority queue more often. We'll then put these two queues in sequence. First the URLs will be placed on priority queues, then placed on politeness queues. See design below.

HTML Downloader

Given a URL, this step makes a request to DNS and receives an IP address. Then another request to the IP address to retrieve an HTML page.

There exists a file on most websites called robots.txt that is used by crawlers as a guide to which pages a crawler is allowed to parse. See below for a snippet of what the https://www.apple.com/robots.txt file looks like. Before attempting to crawl a site, we should make sure to check for this file and follow its rules. This robots file is also a good candidate for cacheing to avoid duplicated downloads.

Disallow: /*retail/availability*
Disallow: /*retail/availabilitySearch*
Disallow: /*retail/pickupEligibility*
Disallow: /*shop/sign_in*
Disallow: /*shop/sign_out*
Disallow: /*/shop/1-800-MY-APPLE/*

Up next we'll cover a few performance optimizations for the HTML downloader

Distributed Crawl: Like most strategies to achieve high performance, distributing tasks among several servers instead of one is ideal. We'll have more than one server in the HTML Downloader be responsible for making HTTP requests given a subset of URLs. See below for an updated diagram of our URL frontier plus a distributed HTML downloader.

Cache DNS Resolver: An average response time for a DNS request can be between 10ms - 200ms. When one thread makes a request to DNS, the others are blocked until the first request is finished. So maintaining our own DNS cache(in memory or separate db) is an effective technique for speed optimization. This cache can be updated periodically using a simple cron job.
Locality: Choose to deploy the servers geographically. When the HTML Downloader server is located closer to website hosts this can significantly reduce download time. This can also apply to all other components in the workflow: crawl servers, cache, queue, storage, etc.
Short timeout: Some web servers may respond slower than others, and we don't want our crawl to be slowed down by this. So it's a good idea to set a maximal wait time on the HTML downloader. If a host does not respond within a predefined time, the downloader will skip and move on to other pages.

DNS Resolver

A URL needs to be translated into an IP address by the DNS resolver before the HTML page can be retrieved.

Content Parser

Any HTML page on the internet is not guaranteed to be free of errors or erroneous data. The content parser is responsible for validating HTML pages and filtering out malformed data.

Content Seen?

With billions of pages the crawler needs to search through, we want to be sure we aren't duplicating our work by eliminating pages that have already been processed in the past. This step will check if the HTML content is already in storage, and if it is, will discard. It will do this by making a query to the content storage database.

Content Storage

A storage system for HTML. This can be a traditional relational database. We can improve the speed of checking for duplicate content by storing only a hash or a checksum of an HTML page, instead of having to parse and store an entire HTML page.

URL Extractor

This step parses over HTML and extracts new URLs to process. For example, below is a snippet of HTML from Apple's website. The URL Extractor will find href values and create full URLs.

URL Filter

This step is responsible for excluding certain content types like PDFs or JPGs, and also URLs on blacklists we may have. It finds these content types by lookin for file extensions in URLs.

URL Seen

This step keeps track of URLs that have already been visited before, or that already exists in the URL Frontier. It accomplishes this by checking if the URL is already in the URL storage database. This helps reduce duplicate work and possible infinite loop conditions.

^{This post is an extract from a book titled "System Design Interview – An insider's guide" By Alex Xu.}

Java 8 Stream API on Arrays

Henri Idrovo — Thu, 29 Apr 2021 20:02:02 +0000

Java 8 introduced the concept of functional programming. My first instinct when I need to iterate is to use a for-loop. So here are some common operations on arrays using the stream api. I'll reference this post when I need a friendly reminder. USE MORE STREAMS! You can read more about java streams here. Also I'm using the Java 8 method reference syntax, you can read more about that topic here.

Given an array of strings that represent a phone number, format and print each to console.

String[] phoneNumbers = new String[] {"3125550909", "3125557676", "3125552323", "3125556161", "3125554141"};

String[] formattedPhoneNumbers = Arrays.stream(phoneNumbers)
        .map(value -> String.format("1-(%s)-%s-%s", value.substring(0,3), value.substring(3,6), value.substring(6,10)))
        .toArray(String[]::new);

Arrays.stream(formattedPhoneNumbers).forEach(System.out::println);

Given an array of strings that represent zip codes, filter out unwanted zip codes and print remaining to console.

String[] zipCodes = new String[] {"60640","94102", "60602", "94115", "60647", "94140"};

String[] subsetZipCodes = Arrays.stream(zipCodes)
        .filter(value -> value.contains("606"))
        .toArray(String[]::new);

Arrays.stream(subsetZipCodes).forEach(System.out::println);

Given an array of sentences, count the number of occurrences for each word and print to console.

String[] lines = new String[] {
        "toast for breakfast ",
        "sleepy in chicago",
        "tired in the morning",
        "coffee in the morning",
        "sleepy breakfast today",
        "breakfast in chicago" };

Map <String, Integer > wordCount = Arrays.stream(lines)
        .map(w -> w.split("\\s+"))
        .flatMap(Arrays::stream)
        .collect(Collectors.toMap(w -> w, w -> 1, Integer::sum));

wordCount.forEach((k, v) -> System.out.println(String.format("%s ==>> %d", k, v)));

Given an array of names, and a custom NbaPlayer class, create a new array with elements of NbaPlayer type.

String[] names = {"lebron", "kyrie", "doncic", "davis", "lavine"};

class NbaPlayer {
    String name;
    NbaPlayer(String name){
        this.name = name;
    }

    @Override
    public String toString() {
        return "NbaPlayer{" +
                "name='" + name + '\'' +
                '}';
    }
}

NbaPlayer[] players = Arrays.stream(names).map(NbaPlayer::new).toArray(NbaPlayer[]::new);

Arrays.stream(players).forEach(System.out::println);

That's all for now. Maybe in the next post I'll do more with streams and other data structures. ArrayLists for sure. 👨🏽‍💻

Scale from Zero to Millions of Users

Henri Idrovo — Tue, 30 Mar 2021 18:05:35 +0000

Lets start off with the simplest possible website setup: 1 web server and 1 database server. (yeah this could have been simpler by running both web service AND database on the same server).

Your choice for database type will probably be a traditional relational database like MySQL, PostgreSQL, or Oracle. These are popular and handle most use cases. These represent data in tables and rows, and supports join operations across multiple tables.

Non-relational databases might be the right choice for you if have the following requirements:

Very low latency
Unstructured data
Serialization and de-serialization of data like json or yaml.
Massive amounts of data

Scaling Web Layer: Part 1

Vertical scaling is the process of adding more power like CPU and RAM. Horizontal scaling is the process of adding more servers. Horizontal scaling is more desirable for large scale applications. And to accomplish this, a load balancer is required.

Scaling Database Layer: Part 1

We can use the same principle of horizontal scaling for the database layer. Except not all database servers will have the same responsibility. There will be 'manager' and 'worker' types. The manager server only supports write operations. The worker server gets copies of the data from the manager server and only supports read operations. All operations that make changes (INSERT, DELETE, etc) will be sent to the manager. All read-only operations will be sent to the workers.

With database replication you gain the following advantages:

Better Performance: Most applications require a much higher ratio of reads to writes. So it is best to distribute reads among several servers.
Reliability & High Availability: If one of your databases crashes, you do not need to worry about data loss because data is replicated.

Putting It All Together (so far)

Ok, after successfully scaling both web and database layers, lets take a look at what we got so far.

Cache

A cache is a temporary storage space that stores database results of frequently accessed data in memory. This way subsequent database requests will fetch the cached results, instead of making a round trip from client to database and back. This caching strategy is called a read-through cache. Below are some considerations when using a cache system.

Consider using a cache when data is read frequently but modified infrequently.
It's good practice to implement an expiration policy. Once cached data is expired, it is removed from cache.
Inconsistency between the cache and database can happen because data-modifying operations are not a single transaction.
Mitigating failures by using multiple cache servers and the over-provision of memory.

Content Delivery Network (CDN)

A CDN is a specific type of cache where static content like images and videos are stored on a network of geographically dispersed servers. A client will make a request for something like an image from the CDN server first, and will only make a request to the web server if the content isn't available on the CDN. Below are some considerations when using a CDN.

Cost: CDNs are run by third party providers who charge you for data transfers in and out. Infrequently accessed assets should not be stored here.
Cache Expiration: If the expiration policy is too long, you run the risk of the delivering stale content. If the expiration policy is too short, you run the risk of repeated requests to the origin server.
CDN failures: Your application should be able to gracefully recover from outages and re-route requests to the origin web server.

The architecture of our website is getting complex. Lets take another look at what our design looks like with a database cache and CDN added.

Scaling Web Layer Part 2: Stateless Web Servers

In order to scale our web layer further we need to allow a client to make requests to any web server at any time. Currently if user A logs into their social media by making requests to server 1, user A will need to continue to make requests to server 1 if they want to remain logged in. See below for an illustration of a "sticky session" scenario.

The opposite of the above scenario is a stateless architecture. Where a user's session data is not stored on the web server itself, but a separate persistent storage location like a relational database or s3.

Data Centers

If our website grows at a fast rate and attracts international users, setting up a clone of our core architecture (web + database) in other parts of the world is common practice. A user is then geo-routed to the closest data center. GeoDNS is a DNS service that allows domain names to be resolved to IP addresses based on the geographical location of a user. The relational database that helps us achieve a stateless web layer is not duplicated.

Lets take a look at our architecture now with stateless web servers and multiple data centers.

In the above scenario, in the event of a significant data center outage, we can direct all traffic to a healthy datacenter. There are some challenges that we need to address to achieve this multi-datacenter architecture.

Traffic redirection: GeoDNS can be used to direct traffic to the nearest data center depending on where a user is located.
Data synchronization: Users from two different regions could use two different databases or caches. In the event of a failover, traffic must be re-routed to a datacenter where data is unavailable. A strategy to replicate data across datacenters is required.
Test and deployment: The use of automated deployment tools to quickly and consistently deploy across datacenters is crucial.

Message Queues

A message queue is a software component that decouples two steps of workflow into 'producers' and 'consumers'. Decoupling of components is a preferred architecture for building scalable and reliable applications. The message queue will serve as a buffer and distribute asynchronous requests.

Consider the use case of an application that supports video uploads. Video publishing is not instantaneous. The application will need to preform several steps like cropping, blurring, audio enhancements, and format conversions before it is available to the public. All of these tasks take time to complete. One group of servers act as the producers of video processing tasks, and another group of servers as the consumers of these tasks and will complete the actual work. See illustration below.

Logging, Metrics, and Automation

As a website's popularity grows adding the necessary infrastructure to support logging, metrics, and automation is essential.

Logging: Allows the dev team to investigate and debug errors.
Metrics: Collecting and analyzing metrics that are relevant to the business helps the team gain insights. Metrics such as:
- Host level metrics: CPU, Memory, disk I/O, etc.
- Aggregated level metrics: performance of all layers of the application: Web, Database, Cache.
- Key business metrics: daily active users(DAU), retention, revenue, etc.
Automation: With a system as complex as this, we need to build or leverage automation tools to improve productivity.

Scaling Database Layer Part 2: Sharding

When the amount of data in application no longer fits on one server alone, a common approach is to distribute this data among several servers. Previously we used more than one database server, but one relational table still fit on one server at a time. In this next iteration, we'll essentially split up one table over several servers. Sharding separates large databases into smaller, more easily managed parts called shards. Each shard shares the same schema, though the actual data on each shard is unique to the shard. A hash function is used when accessing data to find the corresponding shard. For this next illustation, we'll use this simple hash function: user_id % 4.

The most important factor to consider when implementing a sharding strategy is the choice of the sharding/parition key. The partition key is one or more table columns that make a unique key that determine how data is distributed. In the illustration below, "user_id" is the partition key.

Sharding is a common approach to scaling a database layer. It does come with its own set of challenges:

Resharding Data: redistributing data on the cluster is needed when a single shard is full.
Celebrity problem: is when a single shard experiences excessive queries compared to the rest of the cluster.
Join Operations: running a join operation on tables that are spread among several servers is not easy. De-normalizing so that these queries can be run on a single table is a possible solution.

Finally the illustration above shows what the architecture looks like with everything put together. Some parts are condensed.

Thanks for reading!

^{Reference: "System Design Interview An Insider's Guide" by Alex Xu.}

Design Patterns: Factory Pattern, Part 2

Henri Idrovo — Sun, 14 Jan 2018 16:26:16 +0000

originally posted on: henricodesjava.blog

Happy new year everyone! Hope you’re excited about a new year of coding, learning, and growing

We last left off on chapter four of Head First Design Patterns: A Brain-Friendly Guide which covers the Factory patterns. In this post we’ll cover the second pattern in the chapter, known as “Abstract Factory Pattern”.

Both of the factory patterns are responsible of encapsulating object creation, in other words, encapsulating the concrete implementation. Factory method pattern accomplished this with the use of Inheritance, the Abstract Factory Pattern accomplishes this using composition.

Lets continue to use our Pizza Store example from part 1. The textbook gives us a hypothetical example were we want to control what type of ingredients are used in all future pizza stores. It is important to note that all stores will use some type of dough, sauce, cheese, and clams. This means there will be several different types of each ingredients. This is were can group them into “family of ingredients”.

What we will do is create a factory class for each one of these regions. A Chicago, New York, and California ingredient factory. And each of these factories will implement a generic ‘PizzaIngredientFactory’ java interface. Lets take a look at the class diagram for this solution. California ingredient factory is not shown for brevity.

It’s a little messy, but take a close look at the relationships in this class diagram. What we see is that each ingredient category (Dough, Sauce, Cheese, Clams) has an interface that then has implementations appropriate to a region. And these implementations come from their respective PizzaIngredientFactory. This is a going point to give a formal definition to our pattern.

Abstract Factory Pattern: Provides an interface for creating a families of related or dependent objects without specifying their concrete classes.

Below is the PizzaIngredientFactory interface.

Below are the NYC and Chicago implementations of this interface.

We aren’t done yet. We’ll need to make changes to the Pizza so that all pizzas only use factory produced ingredients, instead of accepting any string. Below is the abstract Pizza. Take a moment to look back at my part 1 post and notice the changes made. This abstract pizza and all interfaces and implementation of the ingredients is not shown in the class diagram above for brevity.

Below is what the pepperoni pizza looks like. Notice how all the responsibility of choosing a specific type of dough, sauce, and cheese, is left up to the ingredientFactory. Which in this class is the generic interface, and can be passed a concrete implementation in the constructor. The cheese, clam, and veggie pizza look similar to this.

Finally lets take a higher level look at a client’s use of these pizzas. Below is the NYC pizza store.

And here we see the constructor for ‘NYPizzaIngredientFactory’ being set to the generic ‘PizzaIngredientFactory’ data member. Then it being passed to each pizza types constructor. BOOM, all the magic happens in the background!

That’s a lot of work and the class diagram can look a little confusing, but that’s what it takes to get an implementation of the Abstract Factory Pattern. If you want to take a look at the complete code base for this example check it out here. Thanks for reading

Design Patterns: Factory Pattern, Part 1

Henri Idrovo — Sun, 29 Oct 2017 21:01:50 +0000

Hi everyone. In Chapter four of Head First Design Patterns: A Brain-Friendly Guide we learn about the Factory pattern. The text begins by talking about a programming idiom known as the â€˜the Simple Factory’ as a warm up to two real Factory patterns. The first is known as “Factory Method and the second “Abstract Factory”.

Lets begins by introducing the problem that the factory pattern is solving. When we write code that uses the java keyword new, we are calling constructors to concrete classes and essentially creating a hard dependency on a concrete class. The snippet below shows how a concrete dependency for three different duck classes is created, this can quickly get out of hand. (chapter 1 talked about ducks a lot).

The Factory Patterns can help us clean this up and decouple these dependencies. A new design principle is introduced to help define this design mistake above.

Dependency Inversion Principle: Depend upon abstractions. Do not depend on concrete classes.

Lets give an example to help illustrate the design pattern. I’m going to use the same pizza shop example in the text. Lets say we have our own pizza shop, and we got some code for making pizzas that looks like this.

As soon as we start adding new pizza types, this class can easily get out of hand with _if-statements. _As we learned in earlier chapters we need to identify what part of the this code is likely to change, and what is likely not to change, and separate them.

What the Factory patterns suggest we do is move all the constructors that are likely to change and put them in class of their own. Furthermore, what if our pizza store needs to be able to create groups of different types of pizzas, say if we had to make different kind of pizzas for the New York and another different type for Chicago (see below).

The factory patterns we’ll introduce allow us to organize and decouple the client from the concrete classes. All factory patterns encapsulates object creation, but lets give a formal definition for the first of our factory patterns.

Factory Method Pattern: Defines an interface for creating an object, but lets subclasses decide which class to instantiate. Factory Method lets a class defer instantiation to subclasses.

Lets take a look at a class diagram with the factory method pattern design for our pizza store problem.

First a class diagram for the group known as the “creator classes. These are the classes that are responsible for pizza creation. The abstract creator contains the common code between all pizza creations, that is prepare() bake() cut() box() methods. It also contains the two abstract methods createPizza() and orderPizza(). The concrete creators is where we use the Java keyword new and instantiate classes (known as product classes).

Next we have the Product Classes. These are the classes that we instantiate in the concrete creators. The abstract product serves as a generic type so that the creators don’t need to know what kind of pizza they are making, they just need to know that it is of pizza type.

Now lets take a look at some code for this Factory Method Pattern implementation for our New York and Chicago pizza stores.

Abstract creator:

New York City concrete creator:

Chicago concrete creator:

Next, our group of product classes. First the abstract product. Note that the common code between all pizzas is kept here. Obviously we don’t have real implementation for these, so we simply print to standard output.

I won’t show all the concrete products, but here are a two, for a New York Pizza store, and a Chicago pizza store.

Each concrete product class gets to decide the name, the type of dough, sauce and toppings it has. Here all we see are strings being set, but you get the idea. Then what happens with these ingredients is out of the concrete product’s control (it’s in the abstract product).

Finally lets take a look at a client class making use of this awesome factory pattern. The nyStore and the chicagoStore are our two “factories for New York style and Chicago style pizzas. If you run this code below, you’ll see print statements that describe the preparation with each pizza’s specific ingredients and then each pizzas common baking, cutting, and boxing steps.

If you are interested in looking and running this example, take a look at this repo here. Thats it! This Factory Method pattern relies on inheritance to delegate object creation. My next post will be part two on the topic of Factory patterns where I”ll talk about the Abstract Factory pattern, which relies on composition instead of inheritance. Thanks for reading

Design Patterns: Decorator Pattern

Henri Idrovo — Fri, 06 Oct 2017 17:36:13 +0000

originally posted on my blog at henricodesjava.blog

Hi everyone! Chapter three in Head First Design Patterns: A Brain-Friendly Guide introduces us to the Decorator pattern. A pattern that is heavily used in the JDK’s java.io package. And although not perfect, it can be combined with Factory and Builder pattern (not yet covered) to make them more useful and clean. Let’s talk about the Decorator pattern.

A new design principle is introduced in this chapter. Classes should be open for extension, but closed for modification. And it helps us understand that inheritance is only one form of extension, but not necessarily the best way to achieve flexibility in our designs. The pattern favors composition and delegation to add new behaviors during runtime, rather than during compile-time. In our designs we should allow behavior to be extended without the need to modify existing code. Lets give this pattern a formal definition.

Decorator Pattern: This pattern attaches additional responsibilities to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.

This definition doesn’t really tell us how this pattern is implemented, so lets take a look at a class diagram definition. I wont reproduce the class diagram in the text, I’ll give another slightly different diagram. In the diagram below â€˜Component’ is an interface instead of an abstract class, but the general idea is still the same.

Now let me give a concrete example thats a little different from the text. Lets say we are starting our own taco stand restaurant, and we need to write some code for our taco stand’s ordering system. And lets say our starting point is a â€˜Taco’ interface.

There are four types of tacos we want to serve: beef, chicken, pork, and veggie. Then there are four condiments we can add to a taco: cilantro, onion, avocado, and potato (we don’t put lettuce or tomato on our tacos). Now lets show a class diagram four our taco stand’s ordering system that implements the decorator patten design.

And now lets take a look at some actual code implementing this design. First the Taco interface(component).

Next some Taco implementations (concrete components).

Next lets look at the Condiment Decorator (Decorator).

Now some condiment implementations (concrete decorators).

The Decorator Pattern involves a set of decorator classes(cilantro, onion, avocado, potato) that are used to wrap concrete components (beef, chicken, pork, veggie). So lets take a look at some code that glues this all together. Note that we will learn about much better ways of creating decorated objects when we cover the Factory and Builder design patterns.

Good news. I checked in all this code and verified it works as expected. If you are curious you can check it out on GitHub here.

That is it! That is the Decorator pattern in action, thanks for reading !ðŸ‘‹ðŸ½

Design Patterns: Observer Pattern

Henri Idrovo — Sun, 01 Oct 2017 19:07:20 +0000

originally posted on my blog at henricodesjava.blog

Hi everyone! Chapter two in Head First Design Patterns: A Brain-Friendly Guide is all about the Observer Pattern. They make note of how fundamental and frequently this pattern is used. And also allude to the idea that future patterns in this book are based on this and other pattern combinations. So lets talk about the Observer Pattern.

The problem this pattern is trying to solve is about communication and management. A simple abstract example given in the text is about a newspaper publishing company, and its subscribers. Technically this relationship can be categorized as a ONE-TO-MANY relationship, one publisher, and many subscribers. The text uses the term â€˜ Subject â€˜ to refer to the publisher, and the term â€˜ Observer â€˜ to refer to the subscribers.

Next I’ll give a more concrete example. I’ll use an example slightly different from the text to help illustrate the idea. Lets say we are a finance startup, and we have an idea for a smartwatch app that can communicate with all your financial institutions to give you a bird’s eye view of your finances. For example it can show you balances of your checking account, or show you prices of stocks you own. What we have access to is a class that can return these values for us, lets call this class FinancialData. For the purposes of this example, we don’t care how it retrieves this information.

Below is a naive implementation of our smart watch app, along with an explanation. Of course all of this is just pseudo code, no actual smartwatch code.

There are a few things wrong with this implementation. First we are in violation of a design principle from back in chapter one. _Identify the aspect of your application that vary and separate them from what stays the same. _The lines with the green arrows represent a different screen on the watch, and in the future we might want to add more screens. When that times comes, changes to this class are unfortunately unavoidable. The lines with red arrows are retrieving financial data, and while this too can change, it would be a change caused by a API change and not a change on our client side code. We are also in violation of another design principle: _Program to an interface, not an implementation. _If only there was a way we can neatly separate code that changes, from code that doesn’t, and also not program to an implementation? Introducing, the Observer pattern!

The Observer Pattern : Defines a one-to-many dependency between objects so that when one object changes state, all of it’s dependents are notified and updated automatically.

We will accomplish this, and introduce loose coupling by the use of interfaces. Lets illustrate this definition with a class diagram.

Now with these class diagrams as a guide, lets take a look at an implementation for our smart watch app. First the two interfaces are below.

public interface Subject {
    public void registerObserver(Observer o);
    public void removeObserver(Observer o);
    public void notifyObservers();
}

public interface Observer {
    public void upate(Object obj);
}

Next lets take a look at our new FinancialData class that implements the Subject interface.

public class FinancialData implements Subject {
   private ArrayList observers;

   private float checkingBalance;
   private float savingsBalance;

   private float stockValue;
   private float fourOnekValue;

   private float weekSpending;
   private float monthSpending;

   public FinancialData(){
      observers = new ArrayList();
   }

   public void registerObserver(Observer o){
      observers.add(o);
   }

   public void removeObserver(o){
      int i = observers.indexOf(o);
      if(i > 0){
         observers.remove(i);
      }
   }

   public void notifyObservers(){
      for(int i = 0; i<observers.size(); i++){
         Observer observer = (Observer)observers.get(i);
         observer.update(this);
      }
   }

   public void measurementsChanged(){
      notifyObservers();
   }

   public void setNewBalanceData(float val1, float val2){
      this.checkingBalance = val1;
      this.savingsBalance = val2;
      measurementsChanged();
   }

   public void setNewInvestmentData(float val1, float val2){
      this.stockValue = val1;
      this.fourOnekValue = val2;
      measurementsChanged();
   }

   public void setNewSpendingData(float val1, float val2){
      this.weekSpending = val1;
      this.monthSpending = val2;
      measurementsChanged();
   }
   // other FinancialData methods here
   // including getter methods for balance, investment, and spending values
}

Now lets take a look at one of our observers, the Balance smart watch screen.

public class BalanceScreen implements Observer {
   private float checkingBal;
   private float savingsBal;
   private Subject financialData;

   public BalanceScreen(Subject financialData){
      this.financialData = financialData;
      financialData.registerObserver(this);
   }

   public void update(Object obj){
      if( obj instanceof FinancialData){
         FinancialData fd = (FinancialData) obj;
         this.checkingBal = fd.getCheckingBalance();
         this.savingsBal = fd.getSavingsBalance();
      }
      display();
   }
   // other BalanceScreen methods here
}

Finally lets take a look at a program to glue all this together.

public class SmartWatch {
    public static void main(String[] args){
        FinancialData financialData = new FinancialData();

        BalanceScreen balanceScreen = new BalanceScreen(financialData);
        InvestmentScreen investmentScreen = new InvestmentScreen(financialData);
        SpendingScreen spendingScreen = new SpendingScreen(financialData);

        // not using an actualy finance api, so we set values manually
        financialData.setNewBalanceData(348.12, 3600.87);
        financialData.setNewInvestmentData(899.12, 45000.65);
        financialData.setNewSpendingData(210.34, 677.45);

        // at this point we should see print statements for all 3 screens
    }
}

There you have it! We implemented the interfaces and created some code to test it. Now please note I haven’t actually got this code running, so there might be a bug or two in these lines. For a future post I’ll have to create a project and push this working code up on github. Thanks for reading !ðŸ‘‹ðŸ½

(originally posted at my personal blog: https://henricodesjava.blog)

Design Patterns: Strategy Pattern

Henri Idrovo — Thu, 14 Sep 2017 17:04:07 +0000

originally posted on my blog at henricodesjava.blog

Hello world! I'm currently readingÂ Head First Design Patterns: A Brain-Friendly Guide, and if it is your first time reading about design patterns in Object Oriented Programming(OOP), its a great start! I wouldn't recommend it as a reference guide, because there are a ton of examples, illustrations, and informal descriptions. So here is the first pattern introduced in the book.

Strategy Pattern:Â Defines a family of algorithms, encapsulates each one, and makes them interchangeable. The strategy lets the algorithm vary independently from clients that use it.

Sounds confusing, but it is actually pretty simple. The pattern is based on the following three principles:

Identify the aspect of your application that vary and separate them from what stays the same.
Program to an interface, not an implementation.
Favor composition over inheritance.

Let me try to clear up what these principles mean. I'll use an example that is different from the one included in the book.

Lets say we are starting a robot company. Like Boston Dynamics. We want to build all types of robots. Check out the class below that will be used for all our robot's behaviors.Â

Now here is where we apply our first principle. Our imaginary company needs to be be able to build robots that operate on land, air, or water. All our robots also need to be able to give the greeting, "Hello world!". The greeting is the behavior that stays the same. The other three behaviors are what will change from robot to robot. So we will separate the 'Walk', 'Swim', and 'Fly' behavior from the 'Greeting' behavior. Lets see how.

The second principle states it is better to use interfaces rather than direct implementations. So what we will do is create interfaces for each behavior group that has the possibility of changing. This way there can exist many different implementations of each, see below.

When it comes time to create a specific implementation of a robot, we will extend the Robot class, and in this way have access to the 'greeting' behavior. The greeting behavior can be coded in the Robot class since this will be the same for all robots we ever create. Then when it comes time to add a Walk, Swim, or Fly behavior to our new robot, it will not inherit this behavior from the original Robot class, but it will instead have an implementation of an interface assigned to it, in other words, it will be composed during runtime. This successfully separates these changing, and non-changing behaviors using interfaces. This way we follow our third design principle. Take a look at the diagram below for an overhead view of it all, including three example robots, Dog, Shark, and Eagle robots.

So thats it! That is the quick introduction to the Strategy Pattern. Above you can see what a complete class diagram would look like when we implement the Strategy Pattern. Thanks for reading ðŸ‘‹ðŸ½

(originally posted at my personal blog: https://henricodesjava.blog)

my intro

Henri Idrovo — Sat, 15 Apr 2017 17:27:17 +0000

Hello DEV.TO! My name is Henri and I'm a software engineer from Chicago. I work for HERE Maps(http://360.here.com) and work a lot with java, hadoop, and aws. Catch you all on the wire.

Henri