DEV Community: John Dorlus

Deploying Flask Microservice and React App with Kubernetes and Amazon EKS (Part 1) - Building the Flask API

John Dorlus — Tue, 14 Feb 2023 22:55:45 +0000

In recent years, there has been an increase in the number of hats that engineers are wearing respective to the role that they are in. Front-end engineers are writing more automated tests. Quality engineers are learning how to use and implement Dev Ops Tools. Dev Ops engineers are getting closer to the applications that they are supporting. So, I am creating a little project to get into the Dev Ops side of things. I have had my eye on Docker and Kubernetes and integrating those tools into my development process.

This project is going to consist of several pieces. For phase 1, we will create a Flask microservice for a video game API. This API is going to be very simple. It's going to feature different video game titles, their IGN rating, their Metacritic rating, the title, developer, publisher and platforms that the game can be found on. We are going to use CircleCI to run the Pytest tests as part of the CI pipeline and then deploy with Docker and Kubernetes via AWS.

For phase 2, I will create a React front-end application to display the list of games. We are also going to use Swagger to generate the documentation for the REST API which we will use for the architecture of the UI and the other layers of the application. The front-end will feature automated UI tests while mocking the back end. The tool for the front end tests will either be Cypress, Webdriver.io or Selenium Webdriver. We will deploy these tests via CircleCI as well as part of a separate pipeline but ensure that it can access the other service when deployed.

For phase 3, we will add a persistence layer of either MongoDB, PostgreSQL, or MySQL and deploy the back-end via Kubernetes as a service to tie in the application.

For phase 4 of the application, we will simulate a migration of the application. The application will go from having a RESTful API back-end built in Flask to another type of back end. Some of the possibilities are either a RESTful API being built in another language like Scala or Golang or migrating to something like GraphQL. We can make that choice once we are closer to that step. This is to help understand how to keep an application available while migrating along with seeing how Kubernetes helps with the migration process.

This article focuses on the development of the REST API microservice. We are going to use Flask-Restful to build out our API. Once you create your repository for the application, create a file called api.py which will act as the server.

Flask-Restful is a super popular framework for building out REST APIs because it is very intuitive and you can easily map Python objects and data structures to routes. For this application, we are going to create 2 Python classes within our api.py file. The first is going to represent the object that is a "Game" which will have all of the properties of the game instance that we mentioned earlier.

In this step, we added the Game class which inherits from the Resource class that is built into Flask-Restful. The Resource class is used for the Class object to be mapped to a route and we will be able to overload REST API actions like GET, PUT, POST and DELETE. When we initialize the Game class, we can also set the types for the arguments that we will be expecting for our API. One thing to note is that for the platforms argument, action="append" is there because we can pass in an array of Strings for the platform. This only works in JSON so we are making this API only work for JSON.

For the next step, I added an array of games that will act as our sample data. We will also create a dictionary of all of the parameters and the type they map to using the fields object provided by Flask-Restful. Notice that the platforms maps to a a List of Strings with the line fields.List(fields.String). This is what also allows us to pass an array of Strings for the JSON value of the "platforms" (Image 1.3).

For the next step, we will focus on extending the Game class to be able to retrieve a single game. For all of the actions (GET, PUT, DELETE) we will be searching for a single game first. So we broke that function out into it's own function first called game_search. If the game is not found, then we cannot GET, PUT or DELETE a game so we have the function abort the process with a 404 error. We search for the game by ID and once that is returned, in the GET function we marshal the data in to JSON to provide the response.

We are going to now further extend the Game class to include the PUT and DELETE functions. PUT is to modify an existing record so we search for the game first Once it's found, we iterate through the arguments and if an argument was passed in, it is replaced. Afterwards, the updated game object is marshaled and returned for the response. For the DELETE record, we find the object and then remove it from the list of Games (which we will work on next).

Speaking of the list of games, we are now going to move on to the class that we are going to use that provides the list of games. Also, we will use this to add new games to the list. There will be a GET function and a POST function for this class. We will also make this class inherit from the Resource class from Flask-Restful and provide the same arguments.

The GET function on the GameList class differs from the one on the Game class in the sense that it returns all Games and if there are none, it returns an empty array instead of the 404 error. The POST call creates an ID by checking the list of Games for the highest ID and then adds one to assign to this new Game instance. Then it assigns all of the parameters to their proper keys and adds the new game to the list of games. Finally it returns the marshaled version of the Game instance that was just created.

The final touch to this app is to add the classes we created to their respective routes. The GameList class will correspond to the /games route whereas the Game class will correspond with the Game route which will be expecting an id in the form of an integer. You also have your app entry point which we use port 5050 because port 8000 can be problematic on OSX.

Last, but definitely not least, we have the tests that we wrote using Py.test to test out our routes and ensure we have the proper functionality and responses for our API. This application is the first step in our endeavor to build out a fully fleshed out app white microservices and to deploy them via Docker and Kubernetes. As it stands, this app does not have a UI, no persistence layer and can only be run locally. In our next phase, we will add Swagger UI documentation automation, we will deploy this microservice via Docker and Kubernetes and we will be adding a user interface to this application via React.

Sources and Credit:
Github Code
Cover Image

How Can I Land A Software Job If I Have No Experience

John Dorlus — Wed, 28 Aug 2019 07:45:41 +0000

This is everyone's story, right?

1.) Graduated valedictorian from an A Ranked School and being captain of the Programming Club at 17 years of age.
2.) Graduated Summa Cum Laude from MIT with a Bachelors in Computer Science, minor in Mathematics in 3 years.
3.) Became an intern at Google which converted into a full time role after a year.

This story, as stereotypical as it may seem, is not the be-all when it comes to the background of software engineers. Some of the largest names in the game (Bill Gates, Steve Jobs, Mark Zuckerberg) did not finish college yet led/lead wildly successful careers in tech.

However, the tech scene is competitive and if you don't have the right skills or package them the right way, you may find yourself at the bottom of the pile. What if you don't have any skills yet? What if you can't go to a traditional brick and mortar school? What if you don't have any connections? This post is to provide some suggestions for those who find themselves to be the underdogs of the interview process.

I Got No Skills

The road to being an engineer is a winding road of topics, technologies, projects, collaboration, etc. As an engineer, you will never stop learning. There will always be some new tech on the horizon, some new pattern that's better for creating applications, some new build tool that automates the process better. With that comes the responsibility to continue to hone your craft and add new tools to your toolkit.

Now, at the beginning of journey, making a selection as to what kind of engineer you want to be can be quite daunting. There are Software Engineers, QA Engineers, Test Automation Engineers, Dev Ops Engineers, Data Engineers and many more. It may serve you better to figure out what you want to build and break that down. That can help to build out a pathway of learning.

Example

Future Engineer: I want to build the next Amazon.
Let's break that down:
Amazon is a web site. It is an e-commerce app. It has a catalog of items. For this goal, I can see a couple of skillsets that would be useful. Web Development, Database Administration, API development, just to name a few. Those are possible pathways to pursue.

What About Edjamacation

With so many avenues for education out there, it can be quite daunting to figure out what works best for your situation. There are universities and colleges, MOOCs, Bootcamps, Meetups, Books, one-off online courses, and many other avenues to learn how to build things using code. So which should you do? While I can't make the choice for you, I can give some sample questions that you can ask yourself to help evaluate which form of learning is best for you.

Questions

1.) Can I teach myself effectively?
2.) Do I need the structure of a classroom setting?
3.) Would I qualify for financial aid if I can't afford classes?
4.) Do I need access to professors or tutors if I am having trouble?
5.) How much schooling does my schedule allow for on a weekly basis?
6.) Do I need the assurance of a degree or certificate once I finish my training?
7.) Can I pace my own learning or do I need someone else to pace my learning?
8.) Do I have uninterrupted study time at home or would I need to leave to go to an establishment?
9.) Do I have good enough software or hardware at home or will I need to use public facilities?
10.) Would I be comfortable seeking work myself after my schooling or would I want assistance from my educational establishment?

Of course, there are pros and cons to every type of schooling. I have created a chart from my own personal experience:

Schooling Type	Pros	Cons
Universities/Colleges	Receive Degree. Most expected way to receive education.	May teach out of date techniques and technologies. Expensive. Takes a lot of time and a long time. Necessity to take unrelated courses.
Massively Open Online Courses (MOOC)	Self paced. Taught only what's relevant to track. Open to anyone who can access. No entrance exams. Relatively inexpensive compared to college and bootcamps.	Limited help when trouble arises. Need to be able to teach yourself. Homework can be lacking. Most likely no certificate for completing.
Bootcamp	Consolidated track that still has access to teachers and tutors. Being able to meet other students. May provide certificate of completion. Less $$ than university.	Very condensed program means a lot of time in classroom per week. May not be as respected as College or University. Can be a challenge finding a good one since so many are out there. Inflexible schedule. Can still be expensive for some.
One-Off Online Courses	Self paced. Often has student community online. Usually inexpensive. Frequent course updates. Large catalog of topics to learn. Reviews for courses to aid purchasing decision.	Large amounts of variance between courses. Pre-requisites are not always well defined. Must be able to learn in the provided format.

I Got Skills! Now what?

Show us! Build, dangit! Build like you've never built before! Software development is very interesting because it's one of the few industries where credentials don't really mean too much if you can't prove in the interview process that you can do the work.

It helps immensely if you have some public code or artifacts from said code to share with whoever you are interviewing with. That being said, what should you build? Glad you asked! There are many initiatives that can help if app ideas are lacking.

1.) 30 apps in 30 days (or a variant of that)
Use web search and social media to your advantage. Many people have done the challenge of developing a certain number of apps in a certain number of days (i.e. 30 apps in 30 days) and have documented their journey of doing so. You can take a look at some of the app ideas they had and try your hand at the challenge as well.

2.) Open Source Projects
Many large companies put their code out there for contributors to help out with. GitHub, Microsoft, Facebook, Elastic, Mozilla, etc. These companies have projects that are out in the open with issues being tracked to help those looking to contribute to discover them.

You can also join communities of different projects by looking the project up and finding where they congregate. Could be via IRC, Slack, Discord or other.

This repo is also a good one for finding a project for a good first pull request.

3.) Blog about it!
Tried a new approach to building a web app? Write about it. Spun up your own CI server using a tutorial? Write about it. Finding your college coursework underwhelming? Write about it. If you are doing something of interest....what should you do? Write about it. See how that works? This will help you retain what you learned, articulate yourself better, etc.

Start Interviewing

Once you have put together a portfolio of work that you have done, articles that you have written, projects you have contributed to it's time to start interviewing. In your interviews, you will likely be able to call upon your experiences from building things, blogging things, etc. Even if some interviews don't end in an offer, take those interviews as opportunities to improve. Ask for actionable feedback from the interviewers. Work on exercises that feature the concepts from the coding interviews. Most of all, keep at it! Persistence and refinement through iteration will definitely yield results.

In Conclusion

The road to a job offer can be a challenging one when you have no experience. It's hard, but doable. I have shared a few tips on how to bridge the gap between inexperienced and Jr. Developer. If this helps even one person out there, I consider this late night write up a success.

Building A Review App Using ReactJS, Elasticsearch, App Search and Grommet — (Part 1: Data)

John Dorlus — Thu, 08 Aug 2019 21:38:30 +0000

I have been wanting to build an app for a while now. As a test automation engineer, I get to work with a lot of really cool tech and web apps but as a former web developer, I find myself needing to scratch the itch to build a web application, work with some of the newer JS tech and relearn some of the patterns that have become less familiar to me. To get started, we are going to take some things from the inter-webs.

Our Data

We are going to start off with a sample data set that is found on Kaggle. This data is for different types of ramen noodles that can be found around the world and their ratings. We want to be able to search through these different types of Ramen, sort them, etc.
Let’s get our data going. I don’t want to have to configure an Elasticsearch cluster and everything myself. I would rather have Elasticsearch look at my data and do what’s best for it. This can be done with the App Search tool.

We are going to start in the standard UI for App Search (not the Onboarding tool) and create our first Engine. Let’s call it my-ramen-engine. We are also going to leave the language as Universal.

I took a look at our data and our data came in a .csv file. We need the format to be in .json. So we are going to use an online converter to get our data in the format we want. Once that’s done and we have our ramen_ratings.json file, we will upload it into App Search.

~NOTE~

The .csv file has capitals and spaces in the header row. That threw errors in App Search when I tried to import the data. The way I handled this was to change all fields to lowercase words and use underscores for spaces. For example, I changed Review # to review_number. Once done, we can import the file.

When that’s done, we will be taken back to the Engine Overview page. From there, we want to make some adjustments to the Schema as all fields are imported as text by default. Let’s go ahead and get those updated to reflect their actual values. When we look at our schema, we see that review_number, stars, and top_ten need to be updated to number.

Uh oh! We hit a snag!

We have some data quality issues. Some of our fields have data that cannot be changed to the number type.

There are a few ways we can handle this. We can update the record via the API as outlined on the home page, we can delete the faulty records, or we can fix the data at the source, delete all records and reupload the JSON. I am just going to delete the faulty records since there aren’t very many of them and this is test data. If it were our actual production data, it would make sense to either edit or reimport that data so those records are maintained.

Once this is done, we will be at the documents page. We now have our indexed documents in an Elasticsearch instance that we will be able to make calls to query, etc. App Search will even provide us a nicely packaged React component to just drop into our app! For the next part of the article, I will be diving into creating the actual app using React and Grommit!