DEV Community: Sofia Jonsson

Stop Refreshing Weather Sites: Automate Alerts with Python and Playwright

Sofia Jonsson — Wed, 26 Nov 2025 20:02:54 +0000

Learn how to scrape live weather data, store it in SQLite, and receive automatic notifications - all in a practical end-to-end Python project.

Living in a ski town means the weather basically runs my life. One minute it’s sunny and warm, the next it's below freezing, and we're getting over a foot of snow. More snow = more fun, but I have to be aware that I cannot simply get up and go. Someone has to shovel out the driveway and scrape off the car. Tired of constantly refreshing weather websites, I decided to automate the whole process. I built a Python pipeline that scrapes current conditions, stores them in SQLite, applies simple alert rules, and emails me whenever something important happens, so I never miss a snowstorm (or an exceptionally freezing morning) again.

The goal was simple but practical: create a system that continuously monitors the weather, detects meaningful changes, and notifies me automatically. Here's what it does:

Scrapes weather data from dynamic, JavaScript-heavy sites using Playwright
Extracts key details like temperature, description, and "feels like" values (10-day and snow forecast coming soon)
Stores historical data in SQLite while avoiding duplicates
Applies flexible alert rules (temperature thresholds, snow warnings, etc.)
Sends notifications via email
Runs on a schedule every few minutes, fully automated

This project combines web scraping, scheduling, and database pipelines into one end-to-end system. It's designed to be extensible. You can swap in different data sources, add new alerts, or even integrate a dashboard in the future.

Let's dive into the overall architecture so you can see how all the pieces fit together:

Stack:

Python
Playwright
SQLite
schedule
SMTP email alerts

This is a practical, real-world automation project that I can keep building upon, not just a one-off script. Here's how I built it.

Architecture Overview

Think of it as a mini ETL pipeline:

Extract via Playwright
Transform via parsing + rules
Load into SQLite

Trigger alerts if conditions match

+-----------------+
|  Scheduler      |
| (every 10 min)  |
+--------+--------+
         |
         v
+-----------------+
|  Playwright     |
|  Scraper        |
+--------+--------+
         | HTML
         v
+-----------------+
|  Data Extractor |
| (Selectors)     |
+--------+--------+
         | dict
         v
+-----------------+
|  SQLite DB      |
| (history + dedupe) |
+--------+--------+
         | latest update
         v
+-----------------+
| Alert Rules     |
+--------+--------+
         | yes/no
         v
+-----------------+
| Email Notifier  |
+-----------------+

Each module is swappable—change data sources, database engines, or notification channels without touching the pipeline structure.

Why I Chose Playwright Instead of Requests + BeautifulSoup

Most modern weather sites are not static HTML. They are heavily JavaScript-driven with lazy-loading and pop-ups that break simple scrapers. With requests, you'd get a skeleton page. With Playwright, you get what a real browser renders.

Playwright gives us:

Headless browser rendering
Reliable selectors
Ability to interact with UI elements
Proper async loading

Example Scraper

from playwright.sync_api import sync_playwright

def scrape_weather(url):
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()
        page.goto(url)

        page.wait_for_selector(".CurrentConditions--tempValue--3KcTQ")

        temp = page.locator(".CurrentConditions--tempValue--3KcTQ").inner_text()
        desc = page.locator(".CurrentConditions--phraseValue--2xXSr").inner_text()

        return {
            "temperature": temp,
            "description": desc,
            "url": url
        }

Storing Data in SQLITE

We store only meaningful changes, not duplicates. This keeps the database useful without ballooning it with repeated identical values.

Example Schema:

CREATE TABLE weather (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    temperature TEXT,
    description TEXT,
    feels_like TEXT,
    url TEXT,
    timestamp DATETIME DEFAULT CURRENT_TIMESTAMP
);

Simple Duplicate Check

def should_save(cursor, record):
    cursor.execute("""
        SELECT temperature, description FROM weather
        ORDER BY timestamp DESC LIMIT 1
    """)
    last = cursor.fetchone()
    return last != (record["temperature"], record["description"])

Alert Rules

Current logic:

def should_alert(record):
    if float(record["temperature"]) < 32:
        return True
    if "snow" in record["description"].lower():
        return True
    return False

Email Alerts

Uses environment variables so credentials never end up in source control

import os, smtplib
from email.mime.text import MIMEText

def send_email(subject, body):
    msg = MIMEText(body)
    msg["Subject"] = subject
    msg["From"] = os.getenv("EMAIL_FROM")
    msg["To"] = os.getenv("EMAIL_TO")

    with smtplib.SMTP_SSL("smtp.gmail.com", 465) as server:
        server.login(os.getenv("EMAIL_FROM"), os.getenv("EMAIL_PASS"))
        server.send_message(msg)

Example alert:

Snow expected tonight. Feels like 18F.

Running on a Schedule

import schedule, time

schedule.every(10).minutes.do(run_scraper)

while True:
    schedule.run_pending()
    time.sleep(1)

Deployment options:

Systemd service
Docker worker
Cron + logs
VPS

Real-World Problems I Ran Into

Problem	Fix
Blank data due to slow JS	`wait_for_selector()`
Popups breaking scraping	close modal buttons
Class names changed	centralized config
Timeouts during storms	retry logic
DB spam	dedupe logic

This is where it became real engineering, not just scripting.

Example Terminal Output:

Running scraper...
Found weather data: 18F, Light Snow, Feels like 4F
Saving new data to DB...
Alert triggered: sending email...
Done scraping.

Future Enhancements:

SMS alerts via Twilio
Multi-site verification
Dockerized deployment
Dashboard UI
Logging + retry strategy

Conclusion:

This solved a meaningful personal problem while strengthening engineering skills. And it's something I can continue to build upon and customize to my personal needs.

Automated scraping
Data pipelines
Scheduled services
Secure credential handling
Resilient retries and selectors

If you live somewhere where weather impacts your daily life, building a custom automated monitoring system isn’t just fun, it’s genuinely worthwhile.

You can find the full code on GitHub.

Progressive Web Applications: Under the Hood (PWA)

Sofia Jonsson — Mon, 18 Nov 2019 07:51:29 +0000

Welcome back to the continuation of this two-part PWA blog series! I'm stoked you guys also found PWA's interesting and I'm here to finish up the topic. For those of you who have not read the intro to PWAs, check it out here:

Intro to Progressive Web Apps (PWA)

Sofia Jonsson ・ Nov 11 '19

#pwa #mobile #webapplication #beginners

As promised, I'm going to be diving deeper into what really goes on under the hood of a PWA. So just to reiterate, a Progressive Application is a happy medium between a web and a native app and basically delivering a much more efficient and user-friendly experience.

Let's dive into the main three components of a PWA!

Application Shell

In a nutshell, this is where the content gets placed. You only need the bare essentials like navigation and header elements as this portion of the PWA aren't loaded each time it gets launched by the user. It is instead cached once and then stored to be displayed to the user no matter their internet connection. This is the core component that makes the PWA so fast and efficient.

Service Workers

One of the reasons a PWA is so great is its ability to perform with limited to no internet connection. This is where the Service Worker comes into action. The Service Worker is an independent network script that focuses on the background and determines whether or not the content should be downloaded from the cache or server. This is the fundamental reason why PWA's work offline and are able to organize data. The central requirement for installing a Service Worker is the HTTPS server. The HTTPS server stops performing when idle but re-fires the next time it is needed.

Manifest Files

The Manifest File allows you, the developer to create your custom splash screen. For those of you who don't know what a splash screen it is the default page shown until your app has loaded and provided your user a more native-like experience. For many applications, this can consist of a plain white screen but that's no bueno. You want to engage your user and make it feel like they're using an app, not an unfriendly mobile web page.

Now, let's get into some of the other components:

Push Notifications

PWA Push Notifications are pretty comparable to that of Native Apps, which is really really cool! This all works due to the Service Workers and Push API. Through the Service Worker's Push Manager, the user of the App gets a pop-up suggesting subscribing to push notifications. Once the user has agreed to the terms and conditions, their subscription ID is stored in the database. I mentioned earlier how SW's handle data, but following this order of events, the server takes over and pushes new messages and the SW subsequently fires off the push notification to the specific user device.

Installation Banners

Also referred to as the web app install prompt, it is one of the methods used by PWA's to captivate and engage its users. As modern-day web users, our attention span is quite limited as we bounce back from different applications, activities, etc. so in order to create an engaging experience and reel in the attention of its users PWA's offer for users to save the page to their home screen after about 30 seconds of use. This install prompt is supported by several browsers including Opera v. 32+, Chrome v.31+, Firefox v. 589+, etc. This gives users the experience of having downloaded an app without the hassle of having to actually do it.

Different Caching Strategies

This is super important for creating a seamless experience with a limited internet connection. There are a few different options to consider:

Cache

A cache only PWA is optimal for a completely static site. Here, the data is cached during the installation process and there are no requests sent by the Service Worker to the server.

Network

It's almost self-explanatory. This approach does not use caching, instead, the Service Worker exclusively sends requests to the network and if there's no network response the request simply fails.

Cache First:

Here the Service Worker automatically loads the cached data, without even trying to reach the network. The requests are only sent to the network when there is no cached data available. This approach is actually a great blueprint for stashing static information and App Shell resources. Information will then only be refreshed when the actual app version gets updated.

Network First:

Here, the PWA's strategy is to essentially send requests to the network. If there is a response, the app will render the received data. If not, the Service Worker only renders cached data. This strategy is ideal for apps with dynamic content.

Cache Vs. Network

An instance where you would be interested in this approach is for a device with slow disk access. The PWA will send a simultaneous request to the network and the cache. Normally, the cached data will render first. During this time, the network shoots over the renewed data and saves it to the cache. This data is refreshed in the background in order to not interrupt the rendering data in a cache.

Well folks, there you have it. You now have the basic information you need to determine why you would want to dive deeper into Progressive Web applications and what type of Caching strategy you need based on your own specific app needs. I for one am looking forward to refactoring some of my Flatiron School projects to make them more efficient, but also mobile-friendly! There is so much more to this subject so depending on how my to refactor goes I might have to just keep y'all posted.

Intro to Progressive Web Apps (PWA)

Sofia Jonsson — Mon, 11 Nov 2019 01:49:06 +0000

This last week I encountered a job application that asked me for my favorite trend in technology. Naturally, I went and googled top tech trends in 2019 and came across a pretty interesting list. What really caught my attention was Progressive Web Applications or PWA's. This was super interesting to me since I've recently been working on expanding my React Native knowledge for a mobile app project and this seems like a happy modern medium instead of building a pure native app. Sure, there are situations when you need a mobile application, but more often than not I'm clicking the dismiss popup in my browser. Ain't nobody got time to download an app just to delete it later.

So, this week I want to go over PWA's and why I think they're so cool and why you might want to use one!

What is PWA?

This application type was unveiled by Google back in 2015. It's got web and native app traits and lands somewhere in the middle taking the best from both approaches. It was initially only for Android users, but due to the iOS 11.3 update in March of 2018, iPhone users get to join in on the fun.

Features

Responsiveness: Building responsive applications is another trend for 2019 as there is a high need to provide an excellent user experience for all types of screens and platforms
Progressive updates: Sure, it's in the name. But more importantly, it needs to be progressive so that all types of users can interact with their application. This strategy is the core of the PWA and as a result, the most critical content is pushed to the forefront
Offline Use: This is HUGE. This means that users can use this app despite their browser. This type of application can work with a bad internet connection or even offline
Keeps it Fresh: Due to service workers, new updates are downloaded while the app is open and in use
Secure: Are only served through HTTPS to protect the privacy and security of its app's users
Discoverable: Is indexed in search engines and receives better ranks
Installable and linkable: No need for users to install PWA's through app stores. This is great for the developer's as well as they do not need to wait for approval from Play Market or App Store. The application can be sent via a link on the web and can be saved on a home screen instead.
Push Notifications: As opposed to web applications, PWA has access to sending push notifications and pop up and alert you about friend requests, recent news, new messages, etc.

Basically: it takes far less time to load and is fast to use. It's easier to share, doesn't require a download which is great for users. It's much faster to build and therefore cheaper and you don't have to wait for approval to launch on the market or store. Compared to a website it offers superior UX, works more securely offline and is impressively lightweight.

Good Examples

Here are some examples of large companies that have implemented PWA's.

Twitter: Launched PWA in 2017. Since then Twitter has cut average load time down by a third, decreased bounce rate by a fifth and increased their pages per session metric over 65%!
Forbes: Also launched PWA in 2017 with the goal of increasing user engagement and provide a fast personalized experience. It's PWA loads in 0.8 seconds and increased session per user metric by 43% and engagement by 100%
Tinder: Built using React and Redux. PWA is 90% smaller than the previous native app and loads in less than half the time of the native app. etc..

Lot's of stats here but as a general consensus, their PWA's operate on a fraction of their previous load time and have increased user engagement exponentially!

Frameworks

There are a few different options for frameworks when building out a PWA. Angular JS and Polymer which are both JavaScript based. There are also other Progressive web app tools such as React and Lighthouse, Dev Tools, and Workbox which are all valid options for building a robust progressive web app.

What's going on Under the Hood?

I will be diving into this topic in my next blog post to really get into the different elements that are used in web dev to make up the PWA.

Next week I will be covering:

App Shell
Service Workers
Manifest Files
Push notifications
Installation banners
Different Caching strategies

Beginner Tips: Solving Programming Problems

Sofia Jonsson — Mon, 04 Nov 2019 03:59:11 +0000

This week, I want to focus on one of the most important parts of the interview process; the code challenge. Whether you are working on a whiteboard or are able to code over a shared screen environment, being presented with a question you don't know how to approach can be a daunting task. Perhaps you understand the logic and syntax but don't feel confident getting the ball rolling or you are able to follow someone else's code without a problem and rely on the crutch of too much guidance. What you don't want to do is freeze up and stare at the blank screen.
There can be quite a bit of uncertainty of where or how to start so I decided to write a basic guideline to keep you moving forward.

1. Recieve the question

2. Read the problem. Now read it again.

Give yourself a minute to understand the prompt. If you don't understand the problem properly you can't answer it either. Ask questions, you don't want to misunderstand the directions and waste time working in the wrong direction.
Depending on the prompt, your questions might be along the lines of:

What am I passing into this function?
What is the purpose of this function? What will I return at the end?
What datatypes does this array contain?

3. Plan of attack

This is a good time to write down notes on the board. It will allow you to structure your problem and look for patterns and steps you can eventually optimize and refactor. You will have just established what the purpose of the function is and its parameters in the step above, and you can talk through your strategy by listing it off as pseudo-code on the side of the whiteboard, or just making quick comments in your coding environment. The purpose of this pseudo-code is to help you translate it into code. You can do this line by line in the text editor and provide a clear outline of what you are working to achieve.

4. Answer the Question

Now that you have asked your clarifying questions, organized your plan of attack, and begun to write pseudo-code, its time to translate it into your language of choice and debug. In a text editor environment, I find that I put a console.log() if I'm coding in JavaScript, or puts in Ruby after every line as I'm checking the values of my code and seeing if it is operating the way I intended. What's important is that you come up with a solution. Don't be too good for the brute force solution because the sooner you get that up and running you can move on to your refactor.

5. Simplify

If you went with the brute force solution it's a good idea to mention that you are aware that you provided the brute force or naive solution but that you would improve your code through x and y because of z. If you have time, this is your opportunity to simplify and optimize your code with a refactor. Take the time to consider the space and time complexity of your solution and how it would work on a larger scale. If you're feeling a little fuzzy on Big O notation check out my blog post on it:

Basics: Big O Notation

Sofia Jonsson ・ Oct 18 '19 ・ 3 min read

#bigo #complexities #algorithms #datastructure

6. Practice, practice, practice

Learning to code is a journey, the only way we are going to get better at it, interviewing, and trusting our instincts (not relying on others) is through practice. Welcome constructive criticism, feedback, and make sure to redo a problem especially if you had trouble with it. If you have someone to practice with I highly recommend alternating the interviewer/interviewee position and to put yourself in the hot seat. When you encounter a particularly difficult problem and have to resort to looking up the answer I find that it's super helpful to take the time to write it out with a test dataset to see what your algorithm does at each step of the way. That way you get to dive into the problem and understand how it works so that you can remember it and solve it the next time around.

Remember to not be so hard on yourself and remember how far you've come!

Interview Prep: System Design

Sofia Jonsson — Sun, 27 Oct 2019 17:06:07 +0000

One topic I've been thinking about for the last little bit is System Design. Throughout my informational interviews, I've gotten little tidbits of information to help me prepare for the interview process' at said company. One of these bits of information pertained to system design often phrased in the manner of "be prepared to architect Twitter or FaceBook."

What is Systems Design?

According to the University of Waterloo: Systems design is identified as the philosophy, methods, and approaches to solving problems that are intrinsically multi-disciplinary. Through the consideration of the objective and subjective performance requirements, a design solution is constructed to meet the needs of the customer, the user, and society. Some of the main tools include strategies, procedures, and techniques that help in architecting a specific product or project.

Main Steps

Step 1: Understand the goals of the design

Ask clarifying questions to eliminate ambiguity with the interviewer. Who are the users? What is its primary purpose? How will it be used? What are the likely inputs and outputs?

Step 2:Establish the scope of the project

Now that basic understanding has been established, this is a good opportunity to ask questions about the features of the application. It's imperative that what you are suggesting is along the lines with what the interviewer wants before you move forward. What type of clients does this application want to support(mobile, web, etc)? Are we requiring authentication? Do we want to incorporate analytics? Are we integrating with existing systems?

Step 3: Design for the appropriate scale

We need to determine the scale to gauge if our data is compatible and can fit on one machine. What is the read-to-write ratio? How many simultaneous requests should we expect? Average response time? Limit on data we allow users to provide?

Step 4: Start from a high-level perspective, then get into the nitty-gritty

Begin with the end-to-end processes determined from the goals set in step 1. This might cover APIs, backend services, network architecture, data stores, outlining different clients, etc. and how they all join together to meet the requirements of those stated goals. This is where the conversation will end up covering potential performance bottlenecks and decisions about the delegation of responsibilities. Whatever way you approach this step, remember to start simple and build it out from there.

Step 5: DS&A - Data Structures and Algorithms

Runtime and memory complexity are super important in designing software systems. The efficiency and usability of the application (depending on the scale of the project) are completely dependent on a deep understanding and implementation of DS&A.

Step 6: Tradeoffs

Just about every decision you make will have trade-offs. It's important to be able to talk about them in real-time, offer solutions for them and demonstrate that you are aware of the complexities of systems and how they often require compromise and showcase your understanding through the pros and cons of each approach. Remember, there is no correct answer, just be prepared to explain yourself to show the interviewer that you're logical and will use the appropriate tools to get the job done.

The Process

This can be broken down into four main compartments and are best illustrated by this image:

task definition
conceptual stage
design stage
implementation stage

Basic Examples

For the example portion, I am going to keep it simple and stick to Twitter and showcase some smaller examples. As I'm sure you know by now, architecting an entire social media page would be quite extensive and this goes back to step 2 of the Main Steps portion of this blog and establishing the scope. Remember to ask lots of questions so you aren't diving deeper than you need to and wasting interview time.

In this first example, I am using a diagram showcasing how to design Twitters REST API:

Next, the Full Object Model:

https://www.visual-paradigm.com/tutorials/rest-api-design-twitter-example.jsp

I hope this blog post gave you some clarity on the subject and a better understanding of how to tackle this prompt next time you encounter it.

Good luck!

Basics: Big O Notation

Sofia Jonsson — Fri, 18 Oct 2019 07:08:57 +0000

This week I decided to dive into Big O notation, or algorithmic efficiency. I know it can be a bit of an intimidating topic, but the only way to get more comfortable with it is to dive in and practice! This blog post will cover the basics and serve as a refresher/guide for your journey to better understanding them.

What is Big O?

Big-O complexities pertain to the limiting behaviors of a function when the argument leans towards a particular value or infinity. Basically, it's used in Computer Science to illustrate the performance or complexity of an algorithm. This is always done in the worst-case scenario of that algorithm and can be applied to both the time and space required.

As a developer its important to be aware of the worst possible outcome of your code when it's put to the test. Being aware of this helps us developers improve our efficiency and create applications with better performance.
For instance, if you build an application and only expect a few users to visit your site then there's no problem. However, if you were, say Mark Zuckerberg and designing Facebook, then you would definitely be concerned about how your application would run as millions of users worldwide interact with your site. It is important to understand the worst-case scenario of any code that you build. As someone who is self-taught in this area I admit, it can be difficult to know what each of the complexities looks like out in the "real world" so I have prepared examples below.

Let's look at the time complexities shown in the chart above, explaining the algorithms:

O(log n)

"O of log N"

O(log n) is the best-case scenario. A good example of this is a binary search which divides the growth rate in two by every step of the code's execution. Therefore, on a graph, O(log n) will flatten out as the function progresses and outperform linear algorithms as the inputs increase.

O(1)

"O of 1"

O(1) is referred to as constant time due to the fact that despite the size of our input, it will always take the same amount of time to compute things. Constant time is regarded as the best-case scenario for a function. Basically, it's a function without a loop.

O(n)

"O of N"

O(n), also known as linear time references an algorithm whose performance grows linearly, in direct correlation to the size of the input. You can think of this as one loop over a variable.

O(n log n)

"O of N log N"

O(n log n) is referred to as logarithmic time. A common example of this algorithm is the implementation of a sorting function and in this case, we'll look at Merge Sort. Just like our O(log n)'s binary search we will divide and conquer, and break the problem into smaller sub-problems, in order to recursively solve the subproblems and then combine the solutions to the smaller problems to the initial problem.

O(n²⁾

"O of N squared"

O(n^2), also known as quadratic time, pertains to an algorithm whose performance increases exponentially to the input data size. This is commonly seen in algorithms with nested iterations or loops and the deeper you go, the higher your exponent O(n^3), O(n^4)..etc.

O(2ⁿ⁾

"Big O of two to the N"

O(2ⁿ⁾ often referenced as exponential time, is an algorithm that doubles with each addition to the inputted data. The growth curve of this complexity is exponential, slow at first, and then skyrocketing. An example of this is the recursive solution for the Fibonacci sequence.

O(n!)

"O of n factorial"

O(N!)is bad. Really, really bad. Therefore it's hardly ever shown along with the other complexities, but it's always mentioned in the chart just to show you that it still exists.

I'll end the blog post with these useful tables from https://www.bigocheatsheet.com

Common Data Structure Operations

Array Sorting Algorithms

If you need a video explanation, Gayle Laakman McDowell, from Cracking the Coding Interview does a great job in this YouTube video:

Roman To Integer

Sofia Jonsson — Mon, 14 Oct 2019 03:11:11 +0000

Intro

This week I decided to do some whiteboard practice and came across a pretty neat solution in Ruby that I wanted to share. I had previously completed this question in JavaScript but decided to go back and attempt the LeetCode questions in a different language.

For this specific blog post I am focusing on the classic whiteboard question of converting Roman letters to an integer:

Problem

As stated above, there are 7 Roman numerals to keep track of and for most cases, they are usually written largest to smallest from left to right. However, for any instance where the numbers 4 and 9 are shown we need to adjust our output to reflect that subtraction.

For this example, our input will be "XLIV", and our Output will be "44"

JavaScript Solution

First, I'll show you how I solved for JavaScript:

To break it down further, I put some Console.logs in my Repl.it to see the outputs from the algorithm and how it produced the specific solution of 44.

I set Roman numerals and decimals equal to specific values in separate arrays so that I would not have to subtract once I came across a number with a 4 or 9 and could then loop through it.

As you can see, the algorithm went through the problem 2 times before arriving at the solution:

Ruby Solution

Next, I solved for Ruby:

Once again, I put some "puts" statements in my Repl.it to see how the algorithm arrived at 44. This solution was initially much longer as I thought I needed to incorporate more Roman Numerals into my hash_map_. Ruby has a super useful function for enumerables called each_cons. This then "Iterates the given block for each array of consecutive elements." I put the "puts" statements in there to show how each time it loops through, it looks to the if statement and produces an output for the appreciating and depreciating sums as displayed in the Repl.it as :sum1 and :sum2.

This time around, the algorithm went through the Roman letters 3 times before arriving at the solution:

Time and Space Complexity

The time and space complexity between these two answers were displayed in my submission:

As we can see, the JavaScript solution was significantly more costly than the Ruby solution.

Conclusion

There are so many different ways to solve these LeetCode questions and this is simply an example of two different ways to do it. It's easy to overthink your answer as you're going through the process of trying to solve the problem, but it's good to struggle for a bit and see what you can extract from memory. As you're practicing it's also important to be mindful of how much time you spend trying to solve a problem so that you don't burn yourself out on something so small.

Hope this was useful, good luck practicing these questions!

Adding PDF to React Site

Sofia Jonsson — Mon, 07 Oct 2019 04:37:32 +0000

Intro

I'm currently working on my portfolio site and want to incorporate a PDF version of my resume. To do so I made a copy of my resume, convert it into a PDF and then dragged it into the React components folder of my project. I tried several tutorials to try to simply get a PDF to render on-screen and finally came across a solution which I am here to share with you today.

If you're looking to generate a PDF document in React, look no further than React-PDF.js and Logan Bittner's detailed blog post on How to Build a PDF Viewer with React and PDF.js. I was then able to take what I learned from this tutorial and incorporate it into my personal project.
https://www.pdftron.com/blog/react/how-to-build-a-react-pdf-viewer/

What is PDF.js?

It is a JavaScript library that displays Portable Document Format(PDF) files through HTML5 Canvas. It is a general-purpose, web standards-based platform for parsing and rendering PDFs and can work as a part of a website or a browser.

Outline

These guidelines are as stated by Logan's blog
*If starting a new project

Install Dependencies*
Create Project*
Setup Project
Define the Backend
Invoke pdf.js
Implementation of Webviewer

Diving In

If you are starting a new project, make sure to install your dependencies before moving on and starting your local server. Next, we move on to the project setup. The blog post guides you through the setup of your root file and the creation of your first component. As I progressed, I incorporated this new component in my Linked routes, alongside my other specified pages so that I could direct my user to this PDF viewer. Functionality gets added as we define our backend and we are able to pass a class to implement these functions to the component. By creating separate backend classes to assist the component to render the PDF we make our code more reusable and adaptable for future changes. This is the alternative I was looking for to hard-coding my resume. To implement pdf.js, we need to download the library from https://mozilla.github.io/pdf.js/getting_started/#download and derive contents inside our public folder. If you are confused about how your files should be structured at this point, you can refer to Logan's:

and mine:

In order to add features such as page manipulation or real-time collaboration, we implement the Webviewer library. Once again, download the contents from https://www.pdftron.com/documentation/web/get-started/manually/ and extract them to your public file (as seen in my file structure above). This isn't necessary and your PDF will render without it, but it'll make all the difference aesthetically and user-wise. This is where I decided to stop as I did not want a PDF seal on my resume and I did not need the extended functionality of the extended backends that Logan goes over at the end of his article.

Reflection

Incorporating the code from the tutorial into my pre-existing project was super easy. Finding a tutorial that actually did what it set out to do was by far the hardest part.
By creating a simple route:

I was able to create a separate route for the PDF that I could Link to and that could be reached through the click of an icon.

Good luck, and have fun!

Rails Refresher

Sofia Jonsson — Sun, 29 Sep 2019 18:53:42 +0000

This blog post is intended to be used as a refresher for basic Ruby on Rails Concepts. This might be helpful for someone who learned Rails a while ago and finds themselves asking "what was that one command again...?"
So, first off..

What is Ruby on Rails?

Ruby on Rails is a web-application framework that includes everything needed to create database-backed web applications according to the Model-View-Controller(MVC) and implementing a RESTful design. Rails has two main guiding philosophies: Don't Repeat Yourself (keep it DRY) and Convention over Configuration. What this means is that you want to keep your code as concise as possible. Writing lots of unnecessary code results in WET code (We Like Typing) which honestly nobody likes. Rails also has its own opinion over the best ways to do things in a web application and default to this set standard of conventions, instead of requiring you, the developer, to specify minutiae through countless configuration files.

Object Oriented Principles

In Ruby there are four principles of OOP:

Abstraction: Keeping the viewed interface as simple as possible
Polymorphism: Using the same method for multiple different objects. Modules and ActiveRecord(AR) are examples of this. They help your code be more dynamic and help you keep it DRY
Inheritance: Models in Rails inherit from AR and provide us with helpful methods like find, sum, where, and average. Also helps keep code DRY!
Encapsulation: Maintaining a barrier wherever possible. In Ruby we utilize Encapsulation when we only use attr_reader. This keeps the user of the application from being able to alter any of the code. When we incorporate attr_writer and attr_accessor we allow outside access. In Rails we use strong parameters to encapsulate and shield our attributes. Without strong params we are subject to outside hacking. By setting a private method like set_user and user_params we can limit @user to only be accessible within the scope of the show, edit, update, and destroy actions. The User will also only have access to changing their :username, :password, :bio, :photo_url. For a UsersController with a follow to follower relationship we would set up it up as so:

Creating a New Rails Project

First, make sure you have Rails installed on your machine, if you are reading this as a refresher, then it should already be there. You can check which version you have by running rails --v and if its something like "Rails 6.0.0" then you're good to continue. To create a new application run the line rails new name_of_new_project. Go ahead and open up your text editor, I personally use atom so I type in atom . and then proceed to run the server in my terminal to see that my initial set up is working rails s. You should be able to see this default image on http://localhost:3000

Generate MVC

The Model maintains the relationship between the objects and the databases through AR, and handles validation, association, traction, etc.
The View, ActionView, is what displays our code onto the webpage. The Controller, ActionController handles the commands for your Views. For instance, if you want to "show" your page, then you need to create a command for it in the controller.

This is where CRUD comes in. On an interactive web-page, you will often find yourself creating features for users to Create, Read, Update, and Destroy content. As an example, you can think about Instagram. If you were to create an Instagram copy site with Rails you would want users to be able to see their posts (Read), upload new content (Create), edit their captions (Update), and delete a comment or post (Destroy).

If you're in a hurry and you know which unnecessary files you want to remove for space complexity, then you can run the incredible rails g scaffold _name_ _attribute_:_type_ and Scaffold your models.
*g is short for generate (you can write it out if fully if you love typing)

However, if you want to do it manually, and were making an application about Movies then you would run the commands:

rails g model Movie name: string rating: integer
rails g controller Movies index show
rails g migration AddBiographyTo Movies biography:text

Note that the controller and migration are written plurally as the Model is singular.

At this point in your setup, you can run rails db:migrate to run the migration and set up your relationships. Make sure your relationships between your models are set up the way you intended before you run the migration.

If you make a mistake and need to "go-back" then you can run rails db:rollback and keep running it until you reach the point you want to continue from. This command is like the ctrl+z or cmd+z of your migrations but use it sparingly and go slow if you do end up needing it!

Associations are connections between models and make common operations much easier in your code. So for our Movie example, we could create another Model of "Directors" where both will have a relationship to AR and we can establish their relationship to each other through association.

The Associations between the models through AR use one of the six:

belongs_to
has_one
has_many
has_many :through
has_one :through
has_and_belongs_to_many

The belongs_to association will be written in singular due to it belonging to one element. The plural has_many pertains to the plural elements in the model. Ex:

Routes

Rails router organizes the URLs and expedites them to a controllers action (Rack application) and generates paths and URLs removing the need to hardcode strings within your Views.
When naming a route make sure to use the plural when defining a route for a resource. Ex: resurces :movies
If you are ever confused about what routes you have, you can always rake/run your routes in the terminal using to show the parameters of your routes.
A resourceful route in Rails maps between the HTTP verbs and the URLs to the controller actions. By convention, each of these actions map to a distinct CRUD operation in a database.
Ex:

I think that covers the basics. There are way more concepts and topics I could dive into, but I might just have to save them for a deep dive blog post!

Staying Productive While Traveling

Sofia Jonsson — Sun, 22 Sep 2019 19:59:16 +0000

If you're like me you have probably found it difficult to balance learning, coding, and everything else that goes on in your daily life when you're on the road and traveling around. I personally struggled with how I was going to keep up with my studies while traveling to Colorado to attend a friends wedding. My main priorities at this time are to network, study algorithms, and code to keep my skills sharp, but I've found this a bit difficult as I'm out of my normal productive work environment.

Staying on track can be difficult, but I found that it boils down to three main points:

1. Schedule time for work

I know this might sound redundant, but when you're out of your normal routine and visiting old friends and attending wedding festivities it can be difficult to find the time for yourself and your work- so plan ahead!
Set time aside each morning to go visit a coffee shop and make sure you designate this for your work. Make the most of your time, think of it as High-Intensity Interval Training for your brain. I'm sure you've heard that you need to look away from your screen every 20 minutes for better eye health, but have you heard of interval focus for productivity? This quote from J.D. Meier, a project manager, and productivity blogger states:

"A 20-minute chunk of time is a very useful slice of time and the productive possibilities are endless if you can sustain your focus. The key is to know that sustained thinking takes energy, and it burns out. To address this, take breaks to recharge and renew. Five-minute breaks are a great way to stay focused"*

2. Leverage your network

You never know who the people you know, know. Yeah, that's a tongue twister, but seriously, I had no clue I knew this many people with connections to people in tech. Depending on where you are and who you know this might be easier or more difficult but reach out to the people you know because if you don't ask you'll never know. If you can try to meet up with those people while you're in your set destination.

3. Make a plan for when you return.

It's difficult to achieve everything you plan for during your time away from your normal routine. The most important thing is to keep busy, get at least the bare minimum done, and plan to make up for what you have not been able to achieve when you get back. If you can be proactive and plan to make up for what you lost while traveling or on vacation you'll save yourself the anxiety of what you have not been able to achieve since you've already set a plan to make up for it.

4. Have some fun while you're at it

Seriously, don't be so hard on yourself. Coding and learning are important, but make sure to make time for yourself and the people you love while you're out wherever you are as you'll feel more prepared to return to your work if you actually enjoyed yourself as you were gone.

Good luck out there!

Source: https://www.theglobeandmail.com/report-on-business/careers/management/how-to-focus-in-20-minute-bursts/article13327566/

Deploying to AWS - help

Sofia Jonsson — Mon, 16 Sep 2019 01:16:03 +0000

A few months ago, during my Flatiron days, I decided to create a guide to deploying your app to Heroku: https://dev.to/sofiajonsson/deploying-your-app-to-heroku-ob6 Heroku is a great free platform, but what you're not paying for is a quick and responsive website.

I'm in the process of uploading my portfolio site through Amazon Web Services (AWS) using S3 and route 53 and have run into an issue getting my static endpoint to display the uploaded application. I have spent hours browsing blog posts and StackOverflow and there's something I'm not understanding.
Instead of seeing my site I get a 404 NoSuchKey error:

The YouTube tutorial I've been following suggests setting my endpoint to index.html, but that's not working and I'm assuming it's because the file is nested within one of my other React.js files? I also tried with index.js as that would make more sense, but ran into the same issue. When I tried to provide a direct location such as "src/index.js"or "public/index.html" I would get the following error:

From what I've gathered, AWS is great for hosting static sites and suggests using vanilla JS, so could that be the problem?

I'm curious to hear about your experiences uploading to AWS using S3 and route 53 and any tips or tricks you might have. I'm going to keep working on this and hopefully follow up with an outlined post in case anyone else runs into this issue.

Web Scraping for custom API

Sofia Jonsson — Sat, 07 Sep 2019 17:38:03 +0000

Intro

For my final project at the Flatiron School, I decided that I wanted to build a custom Ski resort, forecast, and snow report tracking app. I recently moved back to the Seattle area after spending the last four years in a remote Colorado ski town and decided to build something I would actually want to use.

The idea stemmed from my obsessive stalking of weather in different cities near Seattle as I try to plan for my weekend activities. Where am I going to find the warmest weather? Where will I have most sun? Most importantly, where can I avoid the rain? I keep track of this information within my weather app on my iPhone and decide what my plans are based on that incoming information.

During the winter I like to do the exact same thing, but I'm browsing various websites checking snow reports, ski resort ticket prices, and weather forecasts. I decided to create an application that incorporated all of those information points that I value as a user where I could then "favorite" them and keep track of them on my own personal page.

After planning out the details I went looking for online API's both free and paid ones. I quickly realized that no online resource was going to provide me with the exact data that I wanted, let alone get half of it and for a decent price so I decided to teach myself how to create a web scraper.

Scraping

There is a great online resource I came across that guides the developer(you and I) through the process of creating a clean and efficient scraping tool by utilizing Ruby, Nokogiri, and HTTParty. The back-end of my project is coded in Ruby on Rails and I highly recommend watching this 30-minute video to create a basic, yet efficient scraper.

Link to YouTube Video

Source of Data

I sourced my information from a public website, and since my projects are just for fun and for my portfolio I'm not going to run into any copyright issues. I decided to scrape 3 different pages off of skiresort.info and limit my data to North American resorts.

I have linked my project at the bottom if anyone is interested in checking out my scraping file. It's located in back_end_final/scraper.rb I believe I scraped almost 90 snow reports, 500ish forecast reports, and almost 1300 resorts for my project. By inspecting the website and targeting the specific id of the element I wanted to be scraped I was able to play around in the terminal until I had every bit of data, down the proper weather icon for the day in my database.

Diving Deep

If you look at the code, you'll notice that my final function resort scraper is filled with ternary statements.

One of the most difficult things that I came across in this process was dealing with incomplete data. skiresort.info hosts so much data on their site that they are unable to keep the amounts available for each resort uniform. Some small ski resort in Canada simply cannot have the same amount of information about it as Whistler or Vail. I dealt with this problem by utilizing ternary statements and diving into the nth-child elements to target the exact data point I wanted for my application.

It's pretty hacky looking and I should probably refactor it, but hey, it works!
Extracting my own data for a project was a lot of fun and definitely see myself using a Web Scraper for my upcoming projects. I realized that Pow Tracker * only * scrapes data from the date that you run the scraping function. Since that makes for a pretty inefficient tracker of data, I would like to set a stretch goal for myself to automate the scraper so that I can have real-time data to work with.

sofiajonsson / back_end_final

Pow Tracker Final Project For Flatiron School Back End

Welcome to Pow Tracker!

What is Pow Tracker?

This app was created for the avid skiier, or ski vacationer to check out North American stats for different ski resorts, their forecasts, and their snow reports. Imagine you are visiting Salt Lake City and want to make the most of your ski trip.. where do you go for the best conditions? You could use a couple of apps, or google your way through a couple of pages, OR you could use Pow Tracker! Pow Tracker pulls live data from the internet to provide the user with current stats on weather, snowfall, what kind of terrain a resort has, and how much each resort costs. As a visitor to the site you can access all these features, but as a user, you will be able to "favorite" a resort, forecast, and snow report and have those stats render on your personalized site…

View on GitHub

DEV Community: Sofia Jonsson

Stop Refreshing Weather Sites: Automate Alerts with Python and Playwright

Learn how to scrape live weather data, store it in SQLite, and receive automatic notifications - all in a practical end-to-end Python project.

Stack:

Architecture Overview

Why I Chose Playwright Instead of Requests + BeautifulSoup

Playwright gives us:

Example Scraper

Storing Data in SQLITE

Example Schema:

Simple Duplicate Check

Alert Rules

Email Alerts

Running on a Schedule

Deployment options:

Real-World Problems I Ran Into

Example Terminal Output:

Future Enhancements:

Conclusion:

Progressive Web Applications: Under the Hood (PWA)

Intro to Progressive Web Apps (PWA)

Sofia Jonsson ・ Nov 11 '19

Application Shell

Service Workers

Manifest Files

Push Notifications

Installation Banners

Different Caching Strategies

Cache

Network

Cache First:

Network First:

Cache Vs. Network

Intro to Progressive Web Apps (PWA)

What is PWA?

Features

Good Examples

Frameworks

What's going on Under the Hood?

Beginner Tips: Solving Programming Problems

1. Recieve the question

2. Read the problem. Now read it again.

3. Plan of attack

4. Answer the Question

5. Simplify

Basics: Big O Notation

Sofia Jonsson ・ Oct 18 '19 ・ 3 min read

6. Practice, practice, practice

Interview Prep: System Design

What is Systems Design?

Main Steps

Step 1: Understand the goals of the design

Step 2:Establish the scope of the project

Step 3: Design for the appropriate scale

Step 4: Start from a high-level perspective, then get into the nitty-gritty

Step 5: DS&A - Data Structures and Algorithms

Step 6: Tradeoffs

The Process

Basic Examples

https://www.visual-paradigm.com/tutorials/rest-api-design-twitter-example.jsp

Basics: Big O Notation

What is Big O?

O(log n)

O(1)

O(n)

O(n log n)

O(n2)

O(2n)

O(n!)

Common Data Structure Operations

Array Sorting Algorithms

Roman To Integer

Intro

Problem

JavaScript Solution

Ruby Solution

Time and Space Complexity

Conclusion

Adding PDF to React Site

Intro

O(n²⁾

O(2ⁿ⁾