I grew up in Oklahoma City. In OKC, you drive everywhere. Public transportation is lacking and it's a huge city — over 100 square miles bigger than Los Angeles.
Ten years ago, I moved to San Francisco and drove nowhere. My car sat on the street as I biked to work. But the Public Works department cleans most streets on a schedule, and if you don't move your car before cleaning begins, they give you a ticket.
The schedule varies from street to street — one block might be Mondays, another Wednesdays — and I would always forget to move my car at the right time. Since I rarely drove it anyway, I ended up selling it.
Fast forward to the present. My wife and I have a child:
And we bought a new car. The memories of those tickets rushed back. I needed something to remind me to move my car at the right time, no matter where I parked. I needed a way to turn this:
into a Google Calendar reminder:
I used iOS Shortcuts and Pipedream to automate this, with help from a handful of APIs. You might not ever park your car in San Francisco, but I hope what I learned about solving a real-world problem will be helpful for any automation you do.
What is Pipedream?
Pipedream is an integration platform for developers. You can use it to run workflows - a set of steps that compose an automation. You can run workflows on HTTP requests, emails, on a schedule, or in response to app-based events like new tweets, new files uploaded to Google Drive, and more.
Within a workflow, you can use pre-built actions that perform common operations across 300+ apps, like creating a new calendar event, or sending an email. You can also write any Node.js code when you need to solve a custom use case where code works best.
You can think of Pipedream as a mix of integration tools like Zapier / IFTTT and serverless platforms like AWS Lambda / Google Cloud Functions.
First attempts - extracting text from the signs
I'll spare y'all the details of my many failed attempts. In short, the real world produces messy data.
Since the street signs include the cleaning schedule, I thought I could take a picture of the sign and use the text to create the reminders. I extracted that text using Google Cloud's Vision API. Then, I parsed that text to get the day and time of the cleaning, creating a Google Calendar reminder. This worked great for a few signs around my neighborhood.
Then I found this sign:
and this one:
Clearly, I couldn't rely on reading the text of every sign. I needed a different approach.
DataSF to the rescue
The City of San Francisco operates an "open data" portal called DataSF. The DataSF team helps source data from various city departments and aggregates it into a single platform.
Anyone can download COVID-19 stats, 311 calls, a map of every tree maintained by the city, and hundreds of other public datasets. All the data are accessible via API.
I'd used DataSF for past projects, so I figured I'd search their portal for "street cleaning". And what do you know - they provide the street cleaning schedule for all city streets.
Every record in this data set had the following structure:
- A Centerline Network Number - the city's unique ID for the street segment (roughly a few city blocks)
- The side of the street this cleaning schedule applies to (different sides have different schedules)
- The day and time of cleaning for that side of the street, e.g. Wednesdays from 12pm to 2pm
- A Well-known text (WKT) representation of the geographic line representing the street segment
That last value is a LINESTRING: pairs of (longitude, latitude) values that compose a line.
LINESTRING (
-122.485667402435 37.778117247851,
-122.485532329198 37.776252458259
)
You can use a WKT viewer like Wicket to visualize these lines on a map:
I wanted to find the street segment closest to where I was parked so I could get its cleaning schedule. I hadn't worked with geo data much, so it wasn't obvious how I'd solve that problem.
DataSF is powered by a platform called Socrata. When you make API requests to DataSF, you're using Socrata's API. They provide a list of functions that can be used in API requests. I scanned this list and found the intersects() function, which "allows you to compare two geospatial types to see if they intersect or overlap each other".
Instead of finding the closest street segments to my car, I could reframe the problem: I want to find the street segment that overlaps with my current location.
I could use my phone to get my current location. But my location is a point, and the street segment is a line in the center of the street. Those geometries don't intersect.
Instead, we need to draw a box around our current location a few meters in each direction. In WKT terms, this box is a POLYGON, and we can ask DataSF whether that polygon intersects with any streets. This also helps address the imperfect accuracy of GPS by broadening the area where we search for close streets.
This worked great, but presented a new problem. Every street segment has two cleaning records: one for the left side of the street, another for the right. The WKT LINESTRING for both sides ran down the center of the street, so there's no way to tell whether you're on the right side or the left side by comparing the distance from your current location. And since GPS isn't perfectly accurate, we wouldn't want a program to pick — it might pick the wrong side.
There's a non-obvious but simple solution to this problem: just let me choose the right time from a list.
Glueing it all together with iOS Shortcuts
Even in the early versions of this project, when I was taking pictures of street signs, I used iOS Shortcuts to kick off the automation from my iPhone.
Shortcuts provides a "no code" programming environment for your iPhone. You can use it to retrieve your current location, take pictures, make HTTP requests, trigger actions in other iOS apps, and more. They even allow for basic flow control (if / then statements, for loops, and more). And they provide a way to prompt a user to select an option from a list. So I used the DataSF API to find the closest streets and their cleaning schedules, then let the user choose the right one.
(Android users should check out Tasker, which offers similar functionality).
End-to-end, here's how the automation works:
- Shortcuts : Get current location
- Shortcuts : Send that location to Pipedream (HTTP Request)
- Pipedream : Get street cleaning schedules for a given location
- Shortcuts : Present schedules in list, let user choose
- Shortcuts : Send chosen schedule and location to Pipedream (HTTP Request)
- Pipedream : Delete old reminders, create Google Calendar reminder with current location
So far, no tickets!
Creating the calendar reminder
This last workflow is neat, and I want to show you two things you might be able to use in other projects.
SFData gave me the day and time of cleanings, but busy streets are cleaned multiple days: Tuesday and Thursday, for example. I had to find the closest calendar date in the future given that schedule. If I parked on that Tue / Thu street on a Wednesday, I'd need to create a reminder to move my car by the next day (Thursday). If I parked there on Friday, the reminder would need to be for next Tuesday.
Enter Chrono, "a natural language date parser in Javascript". Chrono parses "Today", "this Tuesday", and other human-readable dates into a JavaScript Date object. They provide a function that did exactly what I needed:
// Give me the Date object of the first 12PM Wed in the future
// https://github.com/wanasit/chrono#parsing-options
chrono.parseDate("12PM Wed", new Date(), { forwardDate: true });
// Wed Nov 4 2020 12:00:00 GMT+0000 (Coordinated Universal Time)
Once I had JavaScript Dates for streets with multiple cleanings, I compared them and found the closest time. Now we could create a calender reminder.
I quickly found another problem: I might move the car days before street cleaning is scheduled. But that reminder is still on my calendar, and it'd tell me to move my car for the street I parked on days ago. I don't want to manually delete old reminders each time. I needed a way to automatically delete them when I moved my car.
I ended up tagging new calendar events with metadata to identify them as "move car reminders". This way, the Pipedream workflow could find and delete all existing move car reminders before creating a new one. Google Calendar allows you to add this metadata as extended properties:
The fields of the Events resources cover the most common data associated with an event, such as location, start time, etc, but applications may want to store additional metadata specific to their use case. The Calendar API provides the ability to set hidden key-value pairs with an event, called extended properties. Extended properties make it easy to store application-specific data for an event without having to utilize an external database.
In code, you can just attach this metadata in the create event API request:
{
extendedProperties: {
private: {
streetCleaning: true
}
}
}
Finally, I add my wife to the invite, add reminders 24 and 12 hours ahead of the cleaning time, and add the location of the car, which you can view in Google Maps.
Wrapping up
These real-world data problems can be more challenging than typical automations, but they're rewarding to solve. I couldn't have done this without SFData - huge thanks to them for collecting and exposing the cleaning schedules.
Let me know what problems you're automating in the comments!
I wrote this article with VS Code, on my Mac, and published it by running git push. All my DEV posts are tracked by Git, and I can manage them from my local machine. Learn how to do this yourself!
Top comments (2)
Now I remember Tuesday morning in North Beach .. LOL and those evenings when you planned to go out, drove past your house and saw a Prime Parking Spot, quickly pulled in and nabbed it .. then decided not to go out!
lol yes especially in a place like North Beach, can't let an empty parking spot go by.