DEV Community: Doug Black

Find A String in a Massive Git Repo

Doug Black — Wed, 08 May 2019 16:36:28 +0000

Figure I would share this, as I just needed to do this.

We have a massive git repo for an Engauge Analytics project we are working on. Tons of these branches have active work being done, and some were just skipped during the pruning process.

I needed to find 10 characters.

Technically, I needed to find the method that contained those 10 characters, but I knew those 10 characters would be the key to finding it.

Let's say you're looking for a string title the magic string in one of your many git branches in your repo. You can use:

git grep "the magic string" `git show-ref --heads`

If the string exists in your repo, anywhere, you'll get something like this:

***c5cd1d8:path/to/file.js:let a = "the magic string"

Once you have that data, it's as easy as finding the branch!

git branch --contains ***c5cd1d8

There may be an easier way to do this. What do you recommend?

Help! I'm a Tech Lead

Doug Black — Mon, 03 Dec 2018 12:53:17 +0000

I recently became the Technical Lead at the company I work for, Primitive Social. We're a smaller software team, but growing somewhat quickly. The job description is a little fluid, as its a new role and I have some privilege to turn it into what it needs to be.

My question for you: What does it need to be

What are the most essential parts of being a tech lead? What do you wish your tech lead would do for you?

And what do you think are some of the most important technical standards you think need to be written? (There are no wrong answers, every answer has value, no matter how "obvious")

CORS, In A Way I Can Understand

Doug Black — Wed, 14 Nov 2018 11:40:57 +0000

Two days ago, I tweeted my frustration:

I was an hour in to diagnosing a tiny browser error that was breaking the entire project: "CORS error, Access-Control nice try, you can't do that, ha ha" (I'm paraphrasing).

Apparently, it resonated with a bunch of friends online.

So it got me thinking: What is CORS and how can I explain it so even I can understand it?

First, What I Learned CORS Was

Cross-Origin Resource Sharing (CORS) is a simple, browser- and server-implemented security protocol that tells which sites can communicate with which other sites.

Basically, if Site A (your site) wants to call Site B, it needs your permission to do so. But, it stops Site B from calling Site C, without your permission, and causing possible damage to your data or site, or stealing information, or any other variety of terrible things.

A great example of this would be an ad fed to you from a third-party source (which you allow) that wants to call www.stealallthispersonsdata.org (which you don't allow). The ad will show, but the data won't be stolen.

That is, if you set CORS up correctly.

And there are some beginner things to set up CORS. Here goes the way I look at it:

My Imperfect Metaphor

My wife and I need to step out of the house and we leave our 10 year old daughter in charge.

Why You Need CORS

We don't give our daughter any instructions on who to let in and what they can do. Stranger knocks on door, daughter lets stranger in. Stranger invites more strangers, robs our house, spray paints the cat, and steals our personal information.

CORS gives a framework of who is allowed in and allowed to call whom.

Setting the Access-Control-Allow-Origin

We tell our daughter, "Don't allow anyone in except for Miss Patty (our neighbor)." A knock at the door happens, and our daughter asks who it is. She then can make a decision based on who we told her to allow in.

This allows you to tell the browser which sites you will allow in.

Setting the Access-Control-Allow-Credentials

We tell our daughter, "Don't allow anyone in except for Miss Patty, or anyone who knows our secret code word."

Typically CORS doesn't include cookies or other authentication in its requests. Setting this to true throws cookies in the request as well.

Setting the Access-Control-Allow-Methods

We tell our daughter, "You can invite one friend over, but they can't paint the wall."

This tells the CORS request to allow which methods of request: GET, POST, PUT, DELETE. So, if you wanted a site to do a GET, but not to update anything, you can do that.

Setting the Access-Control-Max-Age

We tell our daughter, "You can invite one friend over, but they have to leave at 6 PM."

This one is really imperfect. This sets the value in seconds to cache preflight request results, such as the data in Access-Control-Allow-Headers and Access-Control-Allow-Methods headers.

There's More!

CORS is a concept I'm just beginning to understand. Help me fill in the blanks! What's missing here? What can be added to the metaphor?

Things I Learned Building an Analytics Engine

Doug Black — Sat, 20 Oct 2018 23:15:49 +0000

Oh man, am I excited. This side project has been awhile coming.

I just released Engauge Analytics (https://engaugeanalytics.com/), a web analytics engine that allows you to get meaningful data from your website without stalking your users like some other large entities cough cough Google and Facebook cough cough.

What’s more, the paid tiers of Engauge allow you to get full, automated SEO evaluations of your sites, AND machine-learning driven content evaluations that help you know how your audience will react to your content BEFORE THEY VISIT.

It’s the geekiest thing I’ve ever made, and I’m crazy excited for it. But it was PAINFUL to make.

Here are some of the lessons I learned while building it.

Think About What You're Creating Before Building It

I was deep into building the UI for the app, with the analytics engine running and testing. A friend of mine got me a call with a friend who worked on product in a major company.

I walked him through the product from start to finish, using fancy words like "proprietary".

His first question: "So who are you making this for?"

Great question. I hadn't really thought about it.

So, side projects are fun and challenging. The biggest lesson I learned up front: If you're going to market it, think about who you're marketing it to.

Ask For Feedback

Two weeks before release, I invited a number of users to test it in alpha and give feedback. The feedback was awesome, and lead to a lot of common-sense features that I had missed.

Some of it was a little tough to hear. This was my baby, and I was incubating it from scratch and asking folks for feedback on how that baby should be raised was sometimes painful.

But, with every suggestion came an opportunity to grow the app. Not every suggestion got implemented, but some of it showed me that the app would go nowhere without it.

The Data Got Big Quick

So, the way the app is set up, I knew the database would get big quick. I just didn't realize HOW big.

It was so big with the alpha users that it crashed the entire thing. I had to quickly scale, and think about scaling in a much bigger way.

Scaling the server and DB was easy, but I'm still not totally pleased with this. Any feedback for a relational database that scales large and gives quick queries?

Enjoy the Journey

Maybe you've been here, too: "Side project is done, now what's next? Let's build something else! A space monitor API that looks at dark matter, utilizing....webpack!"

This seems to be the first side project in a long while (maybe ever) that hasn't given me the itch to move on immediately. This is built, and I want to see it grow and succeed and scale like crazy.

I have to admit, I'm still really learning about it. I'll post some more, maybe some how-tos, on how I built some of this. But, I know there are still areas of growth in this that I haven't even touched. Do I copyright it? Patent the algorithms? Scale to a different database structure? Multi-tenancy?

Time to keep learning!

Are Google and Facebook Evil?

Doug Black — Tue, 11 Sep 2018 10:07:10 +0000

Just a question I've been pondering lately. I'm working on a side hustle called Engauge Analytics, something like Google Analytics with a machine learning component. But, I tagged this "Analytics Without the Evil".

It was right around the time that the Facebook Cambridge Analytics scandal happened, and I think I coined it on a whim. It also comes at a time when Facebook, Twitter, and Google (well, not Google) are all sitting before the US Congress to discuss foreign influence in elections and other data issues.

So, I guess, my question is....

Are Google and Facebook and the like that use our personal data for their own good evil? Are their actions in these circumstances wrong?

[Entry] Appointment Scheduler

Doug Black — Tue, 08 May 2018 14:51:19 +0000

What I built

This scheduler allows you to create appointments to be scheduled in different rooms. You can create rooms, create appointments to be added directly to the scheduler, move appointments between rooms and time slots on the scheduler, schedule appointments without a time to be added later (drag and drop them on).

The scheduler uses sockets to communicate with others on the schedule, so that appointments time slots lock out when you are scheduling there, or when you are editing an appointment.

Demo Link

Project currently lives here: dev.to Scheduler: scheduler.dougblackjr.com

Note: This app is made for desktop viewing.

Link to Code

Repo: https://github.com/dougblackjr/devto-scheduler

How I built it (what's the stack? did I run into issues or discover something new along the way?)

STACK:

php7.2
Laravel
Vue.js
Redis
Pusher
fullcalendar
fullcalendar Scheduler
toastr
Digital Ocean for hosting

ISSUES:

There were two challenges.

The first was locking the time slots on the calendar, which have no backend model created for it. In order to show someone across the ocean that the time slot is being edited, and to avoid overlap, I used Redis to create a key that specifically targets that slot on that day for those times for that particular resource.

The second, and possibly more challenging, was dealing with time zones. When I had tried this in the past, I relied heavily on the browser to deal with time. But if someone was halfway around the world, the time slots would get all borky. Therefore, I throw pretty much everything at the server in order to deal with timing.

Additional Resources/Info

Here's my random dump of thoughts:

Pusher made this a thousand times easier.
Though I love my friends around the world, if y'all could move to Eastern Standard time, I would appreciate it.
I created this because scheduling is hard. I'm grateful for the already created things that made this possible. Please steal my code and make it better.
Thank you to dev.to for being a positive community and for making this all possible!

I Made a Thing

Doug Black — Sat, 17 Jun 2017 17:51:08 +0000

It's only been 9 months since I have been in full-time web development. Though I have been freelancing for years, doing development as my primary role was an entirely different animal.

We were a smaller team when I started, and I was handed a huge project. We had a content block on our primary site that needed to pull content from our 12 other properties, and update it in real time. And it was all mine.

I spent 3 months developing something in Laravel, a project I called Feedbot. Being my first time jumping into Laravel meant learning the use of Eloquent models, creating custom helpers and services, working with Slack integrations. Since Laravel lives on the bleeding edge of technology, I had to get a new server up and running with php7 and secure it.

This has been my favorite project to date in my role, and it lead me to some strong conclusions.

No One Cares as Much as You Do

As Feedbot was coming alive, a la the Frankenstein monster, I got more and more excited. The day I finished and allowed for some code review was one of the most exciting days I've had a Dev.

It was done and working. I deployed. I was ecstatic! I wanted it to be the talk of everyone there: this important machine was complete! Much like a new parent, I wanted to post pics of my baby, Feedbot, on my social media and inundate people with talk about it.

But there were no balloons. No party. No gigantic loudspeaker announcing, "Feedbot is alive! Rejoice!" Don't get me wrong, people were happy it was up. But it wasn't to the level I had anticipated. In fact, the times people mention Feedbot are the times it stops working.

No one may know the work that happens behind the metaphorical curtain of what you are doing. But YOU will know. You will be able to beam with pride every time you get a click, a notification, an empty error log. And you can still celebrate it, rely on your work, and enjoy it.

And your team and your users can know that you will take care of it. It's your baby, and you want it to grow!

No One Knows the Project Like You Do

...and that's both good and bad.

There was a day a few months after launch that we realized Feedbot had stopped posting. Where content should have been live in an hour, we were missing days of stuff. I quickly jumped into action, and found one of our sites cookies were causing the issue. On 16 different pages.

Three lines of code, and we were back in business.

There's something powerful about knowing the inner workings like no one else. It means that when there's an emergent problem, your team and users can look to you to make it right in a hurry. It also means that you have a little bit of job security.

But that's not good. What if you get hit by a bus tomorrow?

Because you know the project like no one else, it's up to you to document it, and document it well. You have to give your team a roadmap to know what your models are pulling from where, your controllers are controlling, and your views are making visible. Good documentation will mean that your team won't need you to make something right, and that your good work will outlast you. In fact, writing the documentation before you begin developing will ensure you have a clear trail toward your goal.

If you are the only one who knows the project, it will inevitably fail without you. And the only legacy you may leave is legacy software.

No One Grows as Much as You Will

Being a freelancer for my early Dev years lead to some pretty bad practices. One I discovered pretty fast.

One of Feedbot's early database iterations was built how I usually built databases: Terribly. But it worked, and that was a win in my eyes.

My boss took a look, and recommended that we change the database structure to make it faster and bring it up to modern conventions. This meant redoing 90% of the app, and that was devastating (my boss still claims he made me cry that day, but I'm not so sure).

What I did realize pretty quickly was that I was reinventing the wheel in a lot of my current code. I quickly became familiar with Eloquent relationships, and knocked my code down by 800 lines. 800 lines!

I'm newer to the field of Dev, but I find that everything changes all the time. Because we're in a constantly changing environment of tools, languages, frameworks, and hardware, you must constantly be changing and growing. There are people who know more than you, no matter how long you've been doing it, and many see things from a different angle than you. It is up to YOU to glean that information and to use it to grow.

The next time we had a Laravel project, it was much easier for me to make recommendations, to recommend model structures and begin migrations, to write documentation with know-how, and to get the ship back on course when things began falling off. Another Laravel project, Lunchbot (a Slackbot to tell us what's for lunch each day, somehow more lauded than Feedbot ever was), only took 5 days to complete.

Every project you are doing should be increasing your knowledge base, and leading you to become the best developer you can be.

You've made things, too! What did you learn from your process?

Hi, I'm Doug Black

Doug Black — Sun, 28 May 2017 12:24:57 +0000

I have been coding for 7 years.

You can find me on Twitter as @dougblackjr

I live in Philadelphia, the greatest City on Earth.

I work for American Bible Society.

I mostly program in these languages: PHP, Ruby on Rails, JS.

I am currently learning more about gitflow and Vue.js.

Nice to meet you.