DEV Community: simkimsia

DO Hackathon: DjangoGreen - Server Management - Submission

simkimsia — Mon, 11 Jan 2021 06:50:50 +0000

First Submitted at 2021-01-11 15:00 SGT or 2021-01-11 7:00 UTC

Last Posted at 2021-01-11 15:24 SGT or 2021-01-11 7:24 UTC

What I built

DjangoGreen is a Django-based, server management panel for DigitalOcean Droplets

Category Submission:

Built for Business

App Link

https://djangogreen.com

Screenshots

if the gif is not loading on dev.to then open in a new tab
Login and create new account using DigitalOcean

If the gif is not loading properly on dev.to then open in a new tab
Create new droplets easily with just a single click

If the gif is not loading properly on dev.to then open in a new tab
Destroy droplets just as easily with a single click and a confirm dialog

Description

This is to be a MVP for a server management panel for Django developers. They can easily spin up and deploy new Django apps. I got the server management panel done just in time for the submission. Will add more features going forward after the submission.

Link to Source Code

https://github.com/oppoin/django-green

The branch is : upgrade-everything

Permissive License

MIT

Background

I've always a desire to build my own product business. So I decide to scratch my own itch in terms of making it easier to spin up servers and deploy version 1 Django apps.

How I built it

I built using Django 3.1 and started to use this new thing called htmx. Also, I used a python wrapper for DigitalOcean API as well

Additional Resources/Info

Technologies that I used for this project include:

DigitalOcean API
python-digitalocean the wrapper for DO API
DigitalOcean App Platform
htmx 1.1.0
Django 3.1

Feedback always appreciated!

I was fighting the platform and I lost. Now I'm better for it

simkimsia — Sat, 26 Dec 2020 14:14:35 +0000

Last modified 2020-12-26 22:20 SGT

It was after an email from the DO support followed by a Xmas eve dinner that I realized, I was fighting a losing war.

I started this hackathon insistent on using Django inside Docker because that's how basically I made my own living. You see, I write quotation sofware for a living and my main framework of choice is Django. The issue is that the client insists on deploying it on-premises.

I needed to reduce uncertainty. Uncertainty such as when the dev and production environment drifted too much. A key decision I made for all my Django apps is always run Django inside Docker. Regardless for development or production. It was a good decision because it reduced the number of moving parts.

I carried over that attitude into this hackathon and that was a mistake.

The blindspot

I did notice from the various information sources that yes, DOAP has a standard way of deploying Django app directly within the App Platform without Docker. And the sources were plenty. From sample code repositories to tutorials, and even the community questions.

Yet, I persisted with running Django inside Docker. I also came across a tutorial on running Docker container inside App Platform. So I was sure my way would have no issues.

Boy, was I wrong.

There's no Nginx needed for DOAP

I have to give up my approach eventually. What turned it around was when I faced issues running collectstatic command inside the main App Platform component.

Don't get me wrong. The command works when I run it via the console inside the main component.

But here lies the rub. The browser still cannot access the staticfiles regardless.

The reason for this mysterious behavior is the way App Platform is setup. App Platform doesn't require your app to run its own Nginx or any other web server software because your app utilizes the DigitalOcean's global CDN.

App Platform is a PaaS (Platform as a Service). Therefore, it tends to be more prescriptive on how the various components talk to one another. The idea is that DOAP prefers that you spin up a separate component to serve the static files. Hence, the CDN setup and the non-necessity of a Nginx.

Every architectural decision has its implications. With a decision like that, that means somehow the collectstatic command needs to run inside the component responsible for handling the /static route. Not inside the main component where the django is running in.

Since the console provided by DO has access to only the main component, therefore the developer is reliant on other ways to activate the collectstatic command.

Buildpacks contain automated commands

Recall I said earlier about how DOAP allows you to install Django directly into its app components? The way they do it is by using buildpacks (and they look like buildpacks from heroku).

Inside the python buildpack meant to install Django, there is an automatic instruction to run the collectstatic command whenever there's a deploy.

And I've noticed that if you choose to run the main component using Docker, the other related components such as the one for static files tend to inherit this setting. You cannot override it just at the static files serving component. You cannot choose to have the main component run using Docker and the other components run differently using standard buildpacks provided.

In other words, instead of this

My setup is now

Well, at least I figured out in a deep sense (okay, deeper before I began) how the DigitalOcean's App Platform worked.

And I must say, DOAP is highly promising in terms of where they want to go with this. I can see how cool it eventually will become.

Right now, if I were to choose which DO product to use for a small side project, I would still go with the Droplets.

Time spent thus far 🕒

For first-time reader, this is the section where I explicitly share the time spent. This is to set realistic expectations for readers and myself.

This week, I have spent much more time on this side project.

So far

Total time so far: 27.8 hr
Dev work so far: 22.95 hr
Writing so far: 4.85 hr

Average per week: ~10 hrs of Dev work, ~2 hrs of Writing

Dec 10 to Dec 19 2020

In this period

Total time this period: 12.95 hr
Dev work this period: 9.1 hr
Writing this period: 3.85 hr

Dec 20 to Dec 26 2020

In this period

Total time this period: 14.85 hr
Dev work this period: 13.85 hr
Writing this period: 1 hr

Date	Work	Time
2020-12-21	Dev	0.75
2020-12-22	Dev	1.5
2020-12-23	Dev	4.2
2020-12-24	Dev	2.75
2020-12-25	Dev	4.65
2020-12-26	Writing	1

2020-12-26

Writing up this post 1 hr

2020-12-25

Upgrade everything: django, postgres, and python and test it locally 1.5 hrs
Abandon using docker to run django on DOAP 3.15 hrs

2020-12-24

1.Run collectstatic in main component does not work . Try all sorts of ways to fix it. 2.25hrs

Try to push the static files to repo to test theory 0.5 hrs

2020-12-23

Adding custom domain, making allauth work with github on localhost 3.1 hrs
Make allauth work on remote 1.1 hrs

2020-12-22

the docker finally works locally 1.5hr

2020-12-21

work on making docker work locally 0.45hrs
then test run the docker running locally spend about 0.3 hrs

More than 9 hours spent on dev work so far in this dohackathon

simkimsia — Sat, 19 Dec 2020 11:14:31 +0000

Last modified 2020-12-19 19:10 SGT

I don't want to write a lot today. Just want to update how I have been spending my time on this #dohackathon. More updates next time.

Time spent thus far 🕒

For first-time reader, this is the section where I explicitly share the time spent. This is to set realistic expectations for readers and myself.

Total time so far: 12.95 hr
Dev work so far: 9.1 hr
Writing so far: 3.85 hr

Date	Work	Time
2020-12-19 Saturday	Writing	0.1 hour
2020-12-19 Saturday	Dev work	0.5 hour
2020-12-18 Friday	Dev work	0.1 hour
2020-12-17 Thursday	Dev work	1.25 hour
2020-12-16 Wednesday	Writing	1.25 hour
2020-12-15 Tuesday	Dev work	2.25 hour
2020-12-14 Monday	Dev work	1.5 hour
2020-12-13 Sunday	Writing	0.5 hour
2020-12-13 Sunday	Dev work	2 hour
2020-12-12 Saturday	Writing	2 hour
2020-12-12 Saturday	Dev work	1.5 hour

2020-12-19 Saturday

Separate requirements.txt into base.txt, local.txt, and production.txt (0.5 hr)
Write up time spent (0.1 hr)

2020-12-18 Friday

Make code changes to include the github provider using allauth (0.1 hr)

2020-12-17 Thursday

Try out the DO support suggestion on how to customize the dockerfile_path. Failed 🛑
Install django-allauth in preparation to add github login to my app. Success ✅
Try out the theory on using doctl to update the app sec and success
Total spent is (1.25 hr)

2020-12-16 Wednesday

Write a 3rd post in the series in DEV (1.25 hr)

2020-12-15 Tuesday

Add a frontpage app, static, and staticfiles folder in codebase to experiment (1.25 hrs)
Make the landing page to work finally (1 hr)

2020-12-14 Monday

Work on environment variables and gunicorn successfully. Fail to get the admin page up though (1.5 hrs)

2020-12-13 Sunday

Trying out Nate suggestion involving Docker and Django on DOAP (0.5 hrs)
Writing second post on DEV (0.5 hrs)
Trying out the codefresh example as suggested by Dikshith (1 hr)
Wrote a new ticket and purchased a managed DB cluster on DigitalOcean (0.25 hr)

2020-12-12 Saturday

Trying out how to deploy from GitHub to DOAP and other engineering stuff (1.5 hrs)
Writing on DEV and figuring out how to write in DEV (2 hrs)

I'm calling it DjangoGreen 3/n

simkimsia — Wed, 16 Dec 2020 15:02:23 +0000

Last modified 2020-12-16 22:55 SGT

With the help of the DigitalOcean customer support, I managed to overcome the biggest obstacle (so far and hopefully stays that way!) for my idea. Which means I can reveal the actual project name I wanted to call it -- DjangoGreen.

The idea for my app goes like this.

You have an idea for a SaaS app.
You chose GitHub, Django, Docker, Stripe, and DigitalOcean for your stack.
You type in the project name and press a button. Two things happen.
You get a working v1 SaaS app instantly on DigitalOcean and its codebase inside a GitHub repo.

That's the idea. Basically, it's an idea-to-v1 instant SaaS builder.

If you want to have gamechanger features in your SaaS, you have to write those yourself. What DjangoGreen is responsible for are the mission-critical, standard SaaS features. DjangoGreen gives you a jumpstart in getting your v1 in front of your potential customers.

What was the biggest obstacle?

I can now deploy a Django v2.2 on DigitalOcean's App Platform (DOAP) using Docker. The biggest risk I thought was that it might be very difficult or impossible to deploy Django via a Dockerfile on DOAP.

I found tutorials on deploying:

Django on DOAP and,
Dockerfile on DOAP.

But definitely none that are Django via Dockerfile on DOAP.

I insist on deploying via Dockerfile because I want a universal format for deployment. GitHub may look and work slightly differently from their competitors. Underneath it all, GitHub and their competitors can work with your code so long as you choose Git. Git is pretty much universal and an open format for version control.

Similarly, I want Dockerfile to be the universal format for deployment. Choosing Docker acts as an enabling constraint too. It prevents DjangoGreen from being too coupled with DigitalOcean's App Platform.

I'm relieved I manage to overcome that obstacle despite the lack of documentation.

So what's next?

I like to find (a) simple solutions (b) to overlooked problems (c) that actually need to be solved, and (d) deliver them as informally as possible, (e) starting with a very crude version 1, then (f) iterating rapidly.

-- Paul Graham at Six Principles for Making New Things

I only wrote down some brief steps for my crude version 1 in my linear app.

The key persona I have for the user of DjangoGreen is the product developer. My notes consist only of the developer flow using v1.

Developer Flow for v1

Login using GitHub (Assume developer has a GitHub Free account and DigitalOcean account)
Type repo name (Assume name is okay)
Press Submit button (Assume this button triggers both creation of GitHub repo and the deployment on DigitalOcean)

This results in the following:

A new GitHub repo with the code
A deployment on DOAP as well
The site has a Coming Soon! landing page

That's pretty much my notes. I do want to add that the deployment on DOAP might not happen so seamlessly.

From the email conversation I have with DO's customer support, each app's first-time deployment may require the developer to manually key in additional info in the DigitalOcean admin panel. Subsequent deploys can happen automatically.

On to today's summary of obstacles and time spent.

Blockers and Papercuts

For the first-time reader, this is a quick recap of the 4 categories of obstacles

🛑 Absolute Blocker: I cannot get past this (yet?) and it's critical to building the app
🚧 Critical Slowdown: I took more than 10 mins to figure this out and it's critical to building the app
🦟 Annoying Mosquito: Took more than 10 mins or unsolvable but not critical
🧘🏻‍♂️ Zen: Solved in less than 10 mins and not critical

🧘🏻‍♂️ Zen
Not any today.

🦟 Annoying Mosquito

Two DEV related markdown issue.

I cannot appear to do indentation for nested numbered list.

Action taken: Accept and move on. 🤷🏻‍♂️

I realize that to have a hero image or cover image, I need to place the cover_image: link_to_image within your triple-dashed lines. Aka your front matter

like this

---
series: Building GoGreen on DigitalOcean App Platform Hackathon Series
cover_image: https://dev-to-uploads.s3.amazonaws.com/i/tn54kn95003u4gzbbvn8.png
---

🚧 Critical Slowdown

The past 2-3 days I have spent hours figuring out why I cannot even get the landing page to work and working with the DO customer support. So this was the critical slowdown.

I had to piece together the exact sequence of manual steps, and make the right configuration and code changes before this was solved. I feel like I can write my own DigitalOcean how-to guide at this point.

The next step is to figure out GitHub login. I will probably go with installing the plugin django-allauth to solve this.

🛑 Absolute Blocker

Not at this point.

Time spent thus far 🕒

For first-time reader, this is the section where I explicitly share the time spent. This is to set realistic expectations for readers and myself.

Total time so far: 11 hr
Dev work so far: 7.25 hr
Writing so far: 3.75 hr

Date	Work	Time
2020-12-16 Wednesday	Writing	1.25 hour
2020-12-15 Tuesday	Dev work	2.25 hour
2020-12-14 Monday	Dev work	1.5 hour
2020-12-13 Sunday	Writing	0.5 hour
2020-12-13 Sunday	Dev work	2 hour
2020-12-12 Saturday	Writing	2 hour
2020-12-12 Saturday	Dev work	1.5 hour

2020-12-16 Wednesday

Write a 3rd post in the series in DEV (1.25 hr)

2020-12-15 Tuesday

Add a frontpage app, static, and staticfiles folder in codebase to experiment (1.25 hrs)
Make the landing page to work finally (1 hr)

2020-12-14 Monday

Work on environment variables and gunicorn successfully. Fail to get the admin page up though (1.5 hrs)

2020-12-13 Sunday

Trying out Nate suggestion involving Docker and Django on DOAP (0.5 hrs)
Writing second post on DEV (0.5 hrs)
Trying out the codefresh example as suggested by Dikshith (1 hr)
Wrote a new ticket and purchased a managed DB cluster on DigitalOcean (0.25 hr)

2020-12-12 Saturday

Trying out how to deploy from GitHub to DOAP and other engineering stuff (1.5 hrs)
Writing on DEV and figuring out how to write in DEV (2 hrs)

Building GoGreen 2/n

simkimsia — Sat, 12 Dec 2020 10:52:23 +0000

Last posted at 2020-12-13 11:35 SGT

I was not playing coy when I refuse to talk about what GoGreen is in my previous post.

I'm just not confident it can be done. A key part of the value proposition involves certain engineering efforts between DigitalOcean and Docker not well documented on the Internet.

I reached out to DigitalOcean support and even offering to pay for paid support to expedite my fact-finding requests.

Turns out DigitalOcean doesn't do paid support. But surprisingly, they got back to me via email pretty quickly (less than 5 hours after I asked) and offered me a suggestion (in significant detail).

I'm impressed with that level of support. I'm comparing it against my experiences using paid support for AWS. And DigitalOcean at this point is outdoing even the supposedly customer-centric Amazon.

Anyway, back to the updates.

What I tried today

Typically, when you ask any support questions to any cloud or platform provider, the first email response tends to give you links to the existing documentation. Then it takes about 3-5 emails stretched over possibly weeks before they realize the question you ask is indeed pretty unique.

This happened to me once this year with AWS. In fact, I even managed to point out errors in their own documentation.

This time, DigitalOcean responded to my initial "Nope, I have read that before and that is not what I want. What I want is to do XYZ. And there's no documentation that covers that exactly." response within 2 hours. And the response was highly competent from a senior engineer in DigitalOcean called "Nate".

Now I don't expect it to be perfect and therefore, I met my first roadblock after trying out the method from Nate.

Which brings us to today's Blockers and Papercuts

Blockers and Papercuts

For the first-time reader, this is a quick recap of the 4 categories of obstacles

🛑 Absolute Blocker: I cannot get past this (yet?) and it's critical to building the app
🚧 Critical Slowdown: I took more than 10 mins to figure this out and it's critical to building the app
🦟 Annoying Mosquito: Took more than 10 mins or unsolvable but not critical
🧘🏻‍♂️ Zen: Solved in less than 10 mins and not critical

🧘🏻‍♂️ Zen

Not any today.

🦟 Annoying Mosquito

Editing the title of a DEV post after it's been published appears to be impossible.

Action taken: Accept and move on. 🤷🏻‍♂️

🚧 Critical Slowdown

Not any today.

🛑 Absolute Blocker

So my blocker from taking the next step is this:

After I tried Nate's method and attempted a deploy, I got a deploy error. Not sure what that means, hence I fired a response to Nate and now praying for an answer.

As you can tell from the name of the log, yes, I'm attempting to do something involving Django for my DigitalOcean App Platform (DOAP) hackathon.

Time spent thus far 🕒

✨ New section!

Given most of us are busy with day jobs or schools, taking on a side project such as this hackathon may lead to unrealistic expectations. So, I felt it would be good to share if we can all share the actual time we spent. Of course, I have to lead by example.

After all, if there's anything 2020 has taught me, it's that compassion towards my own self is important. So here goes:

Overall time spent thus far: 4.5 hrs

Date	Time
2020-12-12 Saturday	4.5 hour
2020-12-13 Sunday	1 hour

Trying out how to deploy from GitHub to DOAP and other engineering stuff (1.5 hrs)
Writing on DEV and figuring out how to write in DEV (2 hrs)
Trying out Nate suggestion involving Docker and Django on DOAP (0.5 hrs)
Writing second post on DEV (0.5 hrs)

Conclusion

Let's hope Nate or any of his colleagues in DO get back to me with a permanent solution in the next 24 hrs. Fingers crossed! 🤞

Building GoGreen 1/n

simkimsia — Sat, 12 Dec 2020 10:44:38 +0000

What I'm building

Codename: GoGreen

I'll reveal what I want to build as soon as I solved the blockers. If I cannot solve the blockers fast enough, I may have to abandon the approach altogether.

What I've done so far

✅ Create an app using DigitalOcean’s App Platform that falls under one of the listed categories (I chose "Build for Biz")
✅ Fill out this form to get a $50 credit for DigitalOcean, good for 60 days from the date of redemption.
✅ Setup a private Github repo
✅ Use one of the following permissive licenses for your code: MIT ✅, Apache, BSD-2, BSD-3, or Commons Clause.
✅ Create the first of N number of DEV posts for this series of DO hackathon 2020-2021 (Well, you're looking at it now, aren't you? 😁 )

Getting GoGreen working in DigitalOcean App Platform

The product I'm thinking of building involves heavy use of Docker. Though I might eventually abandon that if it's too complicated.

And as part of the overall value, I cannot use DigitalOcean's App Platform (henceforth I'm calling it as DOAP. I pronounce it as "DOPE!". As in SOAP but with a D) default of having the Dockerfile being in the root of the repository.

Which means I need to

✅ install doctl and doctl access my DO account
◻️ setup the App Sec as a Yaml file accordingly in my GitHub repository

... amongst possibly more steps.

Blockers and Papercuts

As a typical Singaporean 🇸🇬 , ~~complaining~~ giving constructive criticism is in our blood.

This section covers all the obstacles ranging from absolute roadblocks to minor annoyances I face as I work towards this hackathon.

I'll give 4 categories to describe the potential obstacles I face in each post.

🛑 Absolute Blocker: I cannot get past this (yet?) and it's critical to building the app
🚧 Critical Slowdown: I took more than 10 mins to figure this out and it's critical to building the app
🦟 Annoying Mosquito: Took more than 10 mins or unsolvable but not critical
🧘🏻‍♂️ Zen: Solved in less than 10 mins and not critical

🧘🏻‍♂️ Zen

Now that I think about it... is it worth mentioning non-critical obstacles that are solved in less than 10 minutes? 🤔

🦟 Annoying Mosquito

Cannot use the full GitHub flavored Markdown in DevTo to create checkbox

I mean, what era are we living in already? Anyway, I gave up and used emoji like ✅ and ◻️

Not clear how to trigger series inside DevTo

Is it

series: [series name]
series: ["series name"] which was what's given at the DEV article on this hackathon
series: "series name" OR
series: series name?

Oh, I need to wrap that between two lines of ---?

Don't tell me in words. Show me! 😕

By the way, the answer is, right below where you put the main image, type

--- 
series: Building GoGreen on DigitalOcean App Platform Hackathon Series
---

🚧 Critical Slowdown

Searching for answers on how to change the default path to the Dockerfile in the repo for DOAP

This is way easier provided you know the concepts of doctl and App Sec which I didn't at first 🤷🏻‍♂️

🛑 Absolute Blocker

None so far, and hopefully, none that I'll ever meet.

Conclusion

Up to now, I have spent about 3.5 hours on this. More than half of which is to write this up. Anything to avoid doing real work, am I right? 😁

And since this is a side project, I have to stop now.