DEV Community: Eric Leung

Advanced search on Twitter

Eric Leung — Sun, 07 Feb 2021 10:11:52 +0000

There is an advanced search feature on Twitter. Clicking on this, you see a nice popup and text boxes to fill in.

After you fill in these boxes, Twitter populates the normal search bar with the appropriate text to implement those more advanced features. I couldn't find a list of how you can directly write out these searches, so here I write out how to do advanced Twitter searches without using that popup.

All of these words

The most basic of the searches. Type in all the words you want to be found anywhere in a tweet.

what's happening

Exact phrase

In contrast with the previous search, here you can be specific on the exact placement of words using quotes.

"happy hour"

Any of these words

This is similar to "All of these words", except these tweets don't have to include all the words at the same time. A tweet could contain one, two, or more of these words.

cats OR dogs

Note, the parentheses around searches will ensure your search is isolated from other searches. For example,

"happy hour" (cats OR dogs)

None of these words

You can find tweets without specific words using the minus sign.

-dogs

Hashtags

To search a hashtag, simply write it out with the hashtag in front of the word(s).

#ThrowbackThursday

Language

Use the two-letter code for the language to search for those tweets.

lang:es hola

Account direction of tweet

You can search for tweets from and to particular accounts by using the from: and to: keywords.

from:twitter

to:twitter

Mentioning accounts

Simply add the Twitter account with the @ symbol in front.

@Twitter

Miscellaneous filters

Without and with replies.

-filter:replies
filter:replies

Without and with links.

-filter:links
filter:links

Engagement thresholds

Minimum number of replies, favorites,

min_replies:42
min_faves:42
min_retweets:42

Date ranges

Search within specific time intervals using since: and until:.

since:2014-06-14 until:2016-05-15

Cheatsheet

what's happening                   # all of the words somewhere
"happy hour"                       # extract phrasing
cats OR dogs                       # any combination of words
"happy hour" (cats OR dogs)        # combine search types
-dogs                              # tweets not containing words
#ThrowbackThursday                 # hashtag searching
lang:es hola                       # non-English tweets
from:twitter                       # tweets from an account
to:twitter                         # tweeting at an account
@Twitter                           # they mention accounts
-filter:replies                    # filter out replies
filter:replies                     # filter for replies
-filter:links                      # filter out links
filter:links                       # filter for links
min_replies:42                     # minimum replies
min_faves:42                       # minimum favorites
min_retweets:42                    # minimum re-tweets
since:2014-06-14 until:2016-05-15  # time ranges

I hope this proves helpful and makes searching quicker for you.

Getting RSS feeds from Medium publications

Eric Leung — Sun, 10 Jan 2021 20:08:57 +0000

Lots of publications and good content is still hosted on Medium, like the Netflix Tech Blog.

I wanted to access its RSS feed but there was no subscribe button or RSS icon. But I did notice it was a Medium blog. It turns out there's a way to access it by changing the URL a bit.

Using Netflix as an example, they have a custom domain. So you can access its RSS feed by simply adding feed to the end of it.

https://netflixtechblog.com/feed

There are a number of other variations, such as getting a feed from a specific user or publication on Medium. All those combinations are listed here.

Print fixed fields using f-strings in Python

Eric Leung — Tue, 18 Aug 2020 18:36:55 +0000

Formatting strings in Python is very common. Older styles use a combination of % and .format() to format strings. Newer styles use what are called f-strings.

Formatting strings using f-strings are very flexible. One thing that wasn't clear is how to specify the width of strings.

To create an f-string, you must be using Python 3 and type out something like this.

greet = 'Hello'
print(f"{greet}, World!")
# Hello, World

I ran into a situation where I wanted to format an aligned text. To do so, you can use the syntax used in other Python formatting.

init = 34
end = 253
print(f"You had this much money      : ${init:5}")
print(f"Now you have this much money : ${end:5}")
# You had this much money      : $   34
# Now you have this much money : $  253
#                Spacing width    12345

Note the annotation of spacing width with the numbers 1 through 5 to show the spacing differences.

The basic syntax is

{variable:width.precision}

With this syntax, you can even pass variables for each of those values for more dynamic control.

width = 10
precision = 4
value = decimal.Decimal("12.34567")
f"result: {value:{width}.{precision}}"
# result:      12.35

One last fun thing you have control over is you can align these values using the less than and greater than symbols for right-aligned (>) or left-aligned (<).

place = 'here'
print(f"You can have the word {place:<10}")
print(f"Or you have the word  {place:>10}")
# You can have the word here      
# Or you have the word        here

More reference material on how to format strings can be found here.

Explanation for the notation of linear models

Eric Leung — Wed, 15 Jul 2020 04:37:18 +0000

In the Elements of Statistical Learning book, the chapter on Supervised Learning describes equation for a linear model as a vector (Eq 2.1) and matrix form (Eq 2.2). Both are equivalent.

As a vector is another word for a one dimensional array.

We can write this with parentheses going horizontal like

A = (1, 5, 7, 3, 2)

but we can also write this in a more matrix form with square brackets in a vertical form

\begin{bmatrix} 1 \\ 5 \\ 7 \\ 3 \\ 2 \end{bmatrix}

Now we reach Equation 2.1 below.

Y^=β0^+∑j=1pXjβj^ \hat{Y} = \hat{\beta_0} + \sum_{j=1}^p X_j \hat{\beta_j}

This equation is a fancy way to write out an equation you might see for linear regression like this

Y = -23 + 3x_1 + 3.6x_2

Now going back to our Equation 2.1, what do these variables mean? Here's the equation again.

Y^=β0^+∑j=1pXjβj^ \textcolor{orange}{\hat{Y}} = \hat{\beta_0} + \sum_{j=1}^p X_j \hat{\beta_j}

First, $Y^\hat{Y}$ is the dependent variable value, or output vector. This will contain the values for what you want to predict. The caret symbol ^ here is called a "hat". This a hypothetical value we find from our model prediction. So we can read $Y^\hat{Y}$ as "Y hat".

Y^=β0^+∑j=1pXjβj^ \hat{Y} = \textcolor{orange}{\hat{\beta_0}} + \sum_{j=1}^p X_j \hat{\beta_j}

The next variable we run into is $β0\beta_0$ . This is "the intercept, also known as the bias in machine learning."

On an X-Y plot you have seen in grade school, this is where the line crosses the vertical y-axis. Another way to think about this intercept is the baseline value for your dependent value when your independent variable predictors are all zero.

Y^=β0^+∑j=1pXjβj^ \hat{Y} = \hat{\beta_0} + \textcolor{orange}{\sum_{j=1}^p} X_j \hat{\beta_j}

Now, let's gears to talking about some notation. There is a large "E" looking symbol with some numbers and letters, $∑j=1p\sum_{j=1}^p$ . We call the "E" symbol "sigma" and is a fancy way to say "add these things up".

What things are we adding up? And how? The $Xjβj^X_j \hat{\beta_j}$ is actually a representation of matrices of $X_j$ being a row in the matrix below and $βj^\hat{\beta_j}$ as a single value in the column vector.

Xβ^=(x1,1x1,2⋯x1,nx2,1x2,2⋯x2,n⋮⋮⋱⋮xm,1xm,2⋯xm,n)(β1β2⋮βm) X \hat{\beta} = \begin{pmatrix} x_{1,1} & x_{1,2} & \cdots & x_{1,n} \\ x_{2,1} & x_{2,2} & \cdots & x_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ x_{m,1} & x_{m,2} & \cdots & x_{m,n} \end{pmatrix} \begin{pmatrix} \beta_{1} \\ \beta_{2} \\ \vdots \\ \beta_{m} \end{pmatrix}

Similarly, the $β0^\hat{\beta_0}$ is a column vector:

\begin{pmatrix} \beta_{1} \\ \beta_{2} \\ \vdots \\ \beta_{m} \end{pmatrix}

Multiplying everything together will produce a single column vector, $Y^\hat{Y}$ .

We all do this so that we can run equations like

Y = -23 + 3x_1 + 3.6x_2

this over and over again across multiple sets of values, which are encoded as rows in the square matrices above $X$ .