DEV Community: jeikabu

Cross-posting to Tumblr with AWS Lambda

jeikabu — Mon, 05 Jul 2021 09:21:23 +0000

I really enjoyed making a CLI tool to automate cross-posting blog posts to different places. So, I started looking for another platform to add and Tumblr seemed like a good candidate. Again, rough code is on Github.

Tumblr API

The Tumblr API is well documented. But before you get started you need to register an "application". For "Default callback URL" put a valid URL, you can always edit this later (via Account > Settings > Apps > ✎). This will get you an OAuth "consumer" key and secret (sometimes referred to as "client" credentials).

Some parts of the Tumblr API can be called with just the consumer key. For example, to retrieve published posts:

curl -G https://api.tumblr.com/v2/blog/{blog-identifier}/posts?api_key={oauth_consumer_key}
# PowerShell Example:
Invoke-WebRequest https://api.tumblr.com/v2/blog/rendered-obsolete/posts?api_key=xxx

But most of the interesting ones, like creating a new post, require OAuth.

OAuth

Tumblr's docs are a bit light on OAuth 1.0a details, but their implementation closely resembles Twitter's and there's also an RFC. Twitter calls it "3-legged" OAuth authentication, the RFC calls it "redirection-based" authentication. Whatever you call it, the gist is:

POST https://www.tumblr.com/oauth/request_token with client credentials (i.e. consumer key/secret) -> receive "temporary credentials"
Direct user to https://www.tumblr.com/oauth/authorize website for "resource owner" approval
Once they approve access Tumblr will redirect them to our callback URL -> callback receives "verifier"
GET https://www.tumblr.com/oauth/access_token with temporary credentials -> receive user credentials (i.e. token and secret)

First, I'll go over the CLI client changes, then we'll deal with the callback since it's an excuse to write an AWS Lambda in Rust again. The final resulting user token/secret can be re-used, so this only needs to be done once per account, thankfully.

Client

I initially implemented the gory details of OAuth 1.0a using percent-encoding for string encoding along with hmac and sha-1 for the required signing. But in the end settled on oauth1-request since it made the client much simpler.

First, use client/consumer credentials to obtain temporary credentials:

let uri = format!("{}/oauth/request_token", WWW);
let client_credentials =
    oauth1_request::Credentials::new(&self.consumer_key, &self.consumer_secret);
// Sign request using only client/consumer credentials
let auth_header =
    oauth1_request::Builder::<_, _>::new(client_credentials, oauth1_request::HmacSha1)
        // Can optionally specify callback URL here, otherwise Tumblr will use the application default
        //.callback("https://callback_url")
        .post(uri.clone(), &());
let resp = self
    .client
    .post(uri)
    .header(reqwest::header::AUTHORIZATION, auth_header)
    .send()
    .await?;

let resp_body = resp.text().await?;
// Parse `key0=value0&key1=value1&...` in response body for temporary credentials
let mut resp_body_pairs = resp_body
    .split('&')
    .map(|pair| pair.split_once('='))
    .flatten();
let temp_token = get_value(&mut resp_body_pairs, "oauth_token")?.to_owned();
let temp_token_secret = get_value(&mut resp_body_pairs, "oauth_token_secret")?.to_owned();

Now we have the "temporary credentials". We still need the oauth_verifier that will be sent to our callback URL. To receive this from our callback we'll use AWS Simple Queue Service (SQS).

To interact with AWS in Rust there's the recently announced official AWS SDK for Rust which requires AWS access credentials. To obtain AWS access credentials, in the AWS console click $YOUR_NAME (upper-right) > My Security Credentials > Access keys > Create New Access Key (keep these safe). And then set the following environment variables:

$Env:AWS_ACCESS_KEY_ID="xxx"
$Env:AWS_SECRET_ACCESS_KEY="yyy"
$Env:AWS_DEFAULT_REGION="us-east-1"

Now, continuing with the client:

// Create temporary SQS queue `bullhorn-{temporary token}` to receive oauth_verifier from lambda
let queue_name = format!("bullhorn-{}", temp_token);
let client = aws_sqs::Client::from_env();
let output = client.create_queue().queue_name(queue_name).send().await?;
let queue_url = output.queue_url.unwrap();

// Show "resource owner" approval website in system default web browser
let query = format!("{}/oauth/authorize?oauth_token={}", WWW, temp_token);
let exit_status = open::that(query)?;

The "open" crate makes it easy to show the Tumblr resource owner approval page in a browser. While the user is approving our Tumblr application, we wait for our callback to send the oauth verifier via SQS:

// Receive oauth_verifier from lambda via SQS
let messages = loop {
    let output = client
        .receive_message()
        .queue_url(&queue_url)
        .send()
        .await?;
    if let Some(msgs) = output.messages {
        if !msgs.is_empty() {
            break msgs;
        }
    }
};
// Delete the temporary SQS queue
let _ = client.delete_queue().queue_url(queue_url).send().await?;
let verifier = messages[0].body.as_ref().unwrap();

// Exchange client/consumer and temporary credentials for user credentials
let uri = format!("{}/oauth/access_token", WWW);
let temp_credentials = oauth1_request::Credentials::new(&temp_token, &temp_token_secret);
// Must authenticate with both client and temporary credentials
let token = oauth1_request::Token::new(client_credentials, temp_credentials);
let auth_header =
    oauth1_request::Builder::<_, _>::with_token(token, oauth1_request::HmacSha1)
        // Must include `oauth_verifier`
        .verifier(verifier.as_ref())
        .get(uri.clone(), &());
let resp = self.client
    .get(uri)
    .header(reqwest::header::AUTHORIZATION, auth_header)
    .send()
    .await?;

The user token and secret we receive is valid until revoked (via user's Settings > Apps), so we can store them somewhere safe to avoid having to do this again. We can now use Tumblr on the user's behalf.

Callback

Above covered the important parts of the client. The part of OAuth authentication remaining is our callback that Tumblr's resource owner approval webpage redirects to. We could leave an HTTP server running all the time, but that's a bit silly for something we only need occasionally. This looks like a job for "serverless" and AWS Lambda.

Basic Lambda

As before, we'll use the AWS Lambda Rust runtime to implement our lambda function. MUSL is used to create a fully static Rust binary.

In Cargo.toml:

# Rename binary for AWS Lambda custom runtime:
# https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html
[[bin]]
name = "bootstrap"
path = "src/main.rs"

[dependencies]
anyhow = "1.0"
aws_sqs = { git = "https://github.com/awslabs/aws-sdk-rust", tag = "v0.0.10-alpha", package = "aws-sdk-sqs" }
futures = "0.3"
lambda_runtime = "0.3"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tokio = { version = "1.5.0", features = ["full"] }
tracing = "0.1"
tracing-subscriber = "0.2"

In .cargo/config:

[build]
# Override default build target
target = "x86_64-unknown-linux-musl"

# Needed on Mac otherwise build fails with `ld: unknown option: --eh-frame-hdr`
[target.x86_64-unknown-linux-musl]
linker = "x86_64-linux-musl-gcc"

For starters, put something like the basic example in src/main.rs:

use serde::{Deserialize, Serialize};

#[derive(Deserialize)]
struct Request {
    oauth_token: String,
    oauth_verifier: String,
}

#[derive(Serialize)]
struct Response {
    msg: String,
}

#[tokio::main]
async fn main() -> Result<(), lambda_runtime::Error> {
    let func = lambda_runtime::handler_fn(handler);
    lambda_runtime::run(func).await
}

async fn handler(event: Request, ctx: lambda_runtime::Context) -> anyhow::Result<Response> {
    Ok(Response{ msg: "ok".to_owned() })
}

In order to compile this lambda, we also need to build OpenSSL with musl (based on this post):

### Install pre-requisites
# For Mac/OSX: install command line tools and cross-compiler
xcode-select --install
brew install FiloSottile/musl-cross/musl-cross

rustup target add x86_64-unknown-linux-musl

### Build OpenSSL with musl
wget https://github.com/openssl/openssl/archive/OpenSSL_1_1_1f.tar.gz
tar xzf OpenSSL_1_1_1f.tar.gz
cd openssl-OpenSSL_1_1_1f
export CROSS_COMPILE=x86_64-linux-musl-
# `-DOPENSSL_NO_SECURE_MEMORY` is to avoid `define OPENSSL_SECURE_MEMORY` which needs `#include <linux/mman.h>` (which OSX doesn't have).
./Configure CFLAGS="-DOPENSSL_NO_SECURE_MEMORY -fpie -pie" no-shared no-async --prefix=output_abs_path/musl --openssldir=output_abs_path/musl/ssl linux-x86_64
make depend
# Use `sysctl -n hw.physicalcpu` or `hw.logicalcpu` with `-j` if so inclined
make
# Install required stuff to `output_abs_path/musl/` (exclude man-pages, etc.)
make install_sw
# Set value from `--prefix` above
export OPENSSL_DIR=output_abs_path/musl

### Build Rust lambda function
cargo build --release

We could now create our lambda with the AWS console, but the aws-lambda-rust-runtime docs show how to do it via the AWS CLI.

# Set AWS credentials: https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html
aws configure

# Create package with layout required for custom lambda runtime:
# https://docs.aws.amazon.com/lambda/latest/dg/runtimes-custom.html
# -j/--junk-paths avoids directory structure
zip --junk-paths lambda.zip ../target/x86_64-unknown-linux-musl/release/bootstrap

# Get ARN for desired existing role
aws iam list-roles

# Create new lambda function
aws lambda create-function \
    --function-name bullhorn \
    --handler doesnt.matter \
    --zip-file fileb://./lambda.zip \
    --runtime provided \
    --role arn:aws:iam::01234:role/service-role/bullhorn \
    --tracing-config Mode=Active \
    # Needed with CLI V2 
    --cli-binary-format raw-in-base64-out
    --cli-connect-timeout 10000

# Run the lambda
aws lambda invoke \
    --function-name bullhorn \
    --payload '{"oauth_token": "xxx", "oauth_verifier": "yyy"}' \
    # Needed with CLI V2
    --cli-binary-format raw-in-base64-out \
    response.json

# To update lambda binary
aws lambda update-function-code \
    --function-name bullhorn \
    --zip-file fileb://./lambda.zip \
    --cli-connect-timeout 10000

--cli-connect-timeout deals with commands failing with Error: Connection was closed before we received a valid response (see issue)
Add --region <region> for a region other than the default set with aws configure

API Gateway

To get Tumblr to call our lambda via the callback URL we use AWS API Gateway to trigger our lambda via an HTTP endpoint.

Open the lambda in AWS console then + Add trigger > API Gateway > Create an API, for "API type" REST API ("HTTP" likely also works) and for "Security" Open.

From the list of triggers expand Details and note "API endpoint". This can be set as "Default callback URL" of the Tumblr application as well as the oauth_callback parameter in the client.

When http://tumblr/authorize redirects to the callback you get https://your_callback_url?oauth_token=xxx&oauth_verifier=yyy. Apparently lambdas don't accept query strings (the ?xxx=yyy bits), so we need to move the query into the body.

In AWS console open API Gateway and /bullhorn ANY > Integration Request, deselect Use Lambda Proxy integration and expand Mapping Templates. Select When there are no templates defined, Add mapping template enter application/json and click ☑. Now there's two options:

From "Generate template" pick Method request passthrough. This will result in the input to your lambda containing the entire request:

transformations: {
"body-json" : {},
"params" : {
    "path" : {}
    ,"querystring" : {
        "oauth_verifier" : "yyy",
        "oauth_token" : "xxx"
    }
    ,"header" : {
    }
},
"stage-variables" : {
},
"context" : {
    ...

Create a simple mapping to move the query parameters into the request body:

{
    "oauth_token": "$input.params('oauth_token')",
    "oauth_verifier": "$input.params('oauth_verifier')"
}

The latter is easier. See the docs for template syntax details.

Don't forget to Save!

In API Gateway you can verify everything works, for "Method" GET and "Query Strings" oauth_token=xxx&oauth_verifier=yyy then Test. In the log output you'll see: the original query, the mapped endpoint request body, and the output from the lambda.

Once this is working you must select Actions > Deploy API, set "Deployment stage" to default and then Deploy. Failure to do this will result in your lambda receiving the query string from the initial API Gateway configuration.

CloudWatch Logs isn't enabled by default for API Gateway. To enable logging you'll need to create a role and enable them in API Gateway > {select API} > Stages > default > Logs/Tracing.

Final Lambda

In order to work with SQS queues our lambda needs some additional permissions:

From AWS console open IAM
Select the role used by the lambda function and expand the policy
Edit policy > Add additional permissions
- "Service" = SQS
- "Actions" = Read / GetQueueUrl and Write / SendMessage
- "Resources" = arn:aws:sqs:us-east-1:01234:bullhorn-* (the naming convention used for our SQS queues)

A working version of the lambda is just the send portion of SQS:

async fn handler(event: Request, _ctx: lambda_runtime::Context) -> anyhow::Result<Response> {
    let client = aws_sqs::Client::from_env();
    // Get SQS queue based on name set by client
    let queue = client
        .get_queue_url()
        .queue_name(format!("bullhorn-{}", event.oauth_token))
        .send()
        .await?;

    // Send oauth_verifier to client so it can retrieve user token/secret
    let res = client
        .send_message()
        .message_body(event.oauth_verifier)
        .set_queue_url(queue.queue_url)
        .send()
        .await?;
    let response = Response {
        message_id: res.message_id,
        sequence_number: res.sequence_number,
    };
    Ok(response)
}

Our client will receive the verifier and complete OAuth authentication as described earlier.

Since managed services like SQS generally exhibit eventual consistency, I wonder if get_queue_url() can temporarily fail and needs to be retried. It's worked fine the way it is (so far), so maybe that's accounted for when the client creates the queue?

Posting to Tumblr

Thus far everything was related to OAuth and obtaining user credentials. Once we have them we can interact with Tumblr.

The Tumblr API has both a "legacy" and "neue" forms. Legacy supports markdown, but not canonical URLs. Neue has canonical URLs but not markdown; posts must be composed from Neue Post Format (NPF) blocks. It might work to build the markdown as HTML and embed in a block, but I'll look into that later. Until there's a better solution, we'll create a "link" post that's just a hyper-link to the original article.

// Check if the article already exists
let posts: Posts = self
    .client
    .get(format!("{}/blog/{}/posts", URL, self.blog_id))
    // Only requires api_key authentication, get response in "Neue Post Format"
    .query(&[("api_key", &self.consumer_key), ("npf", &"true".to_owned())])
    .send()
    .await?
    .json()
    .await?;
let existing = posts.response.posts.iter().find_map(|p| {
    p.content
        .iter()
        // Find block that is a "link" and contains canonical URL, and return its ID
        .find(|block| match block {
            ContentBlock::Link { display_url, .. } => {
                display_url == canonical_url
            }
            _ => false,
        })
        .map(|_| p.id_string.clone())
});

// There's a series of structs for serde to parse the JSON response.
// The following are the important ones:

#[derive(Debug, serde::Deserialize)]
struct Post {
    id_string: String,
    content: Vec<ContentBlock>,
    // <SNIP>
}

// Serde will serialize these from JSON like, e.g.:
// {type="link", display_url="xxx", ...}
#[derive(Debug, serde::Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
enum ContentBlock {
    Link { display_url: String, title: String },
}

Create/update an article:

// Must authenticate using both client/consumer and user tokens/secrets
let token = oauth1_request::Token::from_parts(
    &self.consumer_key,
    &self.consumer_secret,
    &self.token,
    &self.token_secret,
);
let tags = post.front_matter.tags.map(|tags| RequestTags { tags });
let request = LinkRequest {
    // If we found existing article this will be Some and we'll update.  Otherwise this is None and we create.
    id: existing.clone(),
    title: Some(post.front_matter.title),
    url: canonical_url,
    tags,
    ..Default::default()
};

// To create an article: POST {blog_id}/post
// To update: POST {blog_id}/post/edit
let uri = format!(
    "{}/blog/{}/post{}",
    URL,
    self.blog_id,
    if existing.is_some() { "/edit" } else { "" }
);
// Sign the request and create `Authorization` HTTP header
let auth_header =
    oauth1_request::post(uri.clone(), &request, &token, oauth1_request::HmacSha1);
// For POST, request body contains `application/x-www-form-urlencoded`
let body = oauth1_request::to_form_urlencoded(&request);

let resp = self
    .client
    .post(uri)
    .header(reqwest::header::AUTHORIZATION, auth_header)
    .header(
        reqwest::header::CONTENT_TYPE,
        "application/x-www-form-urlencoded",
    )
    .body(body)
    .send()
    .await?;

// HTTP request to create/update "link" type post
#[derive(oauth1_request::Request)]
struct LinkRequest {
    /// Must be `Some` when updating an existing article, `None` when creating a new one
    id: Option<String>,
    #[oauth1(rename = "type")]
    r#type: String,
    tags: Option<RequestTags>,

    // <SNIP>
}

// Helper to serialize Vec<_>
struct RequestTags {
    tags: Vec<String>,
}

// Need to impl Display so oauth1_request knows how to serialize a Vec<_>
impl std::fmt::Display for RequestTags {
    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> {
        let tag_param = self.tags.join(",");
        write!(f, "{}", tag_param)
    }
}

The resulting post looks like this:

Yet Another Snazzy Rust CLI

jeikabu — Sat, 19 Jun 2021 16:05:32 +0000

I recently looked at using pjeziorowski's rollout tool to cross-publish to hashnode and devto. Started making a few minor changes. Then considered a few contentious changes. And in the end decided I should just make my own in Rust. Because... Rust in all the things.

I settled on the following requirements:

Post multiple places

I've been using GitHub Pages and cross-posting to devto via RSS feed. But this tends to munge articles a bit; strips syntax highlighting, adds a linefeed to end of code blocks, mangles links to other articles, etc. All this requires manual fixing before publishing, so I've never been happy with it.
Keep articles up to date

POSTing is one thing, but I regularly make updates/corrections that I'd like to propogate. Pretty much all my devto articles are out of date, for this I need PUT.
Manage canonical_url (and front matter, in general)

Based on where/when the original is posted set the canonical URL for the cross-posts.
Async

I'm impatient. But I also anticipate needing to do a few requests to publish and update articles.
Markdown

Articles are written in markdown and may contain Jekyll front-matter as used by Github Pages.

To avoid confusion with the original (or some Ruby feature flag thingie), I dub thee... BULLHORN (stylized in all-caps, for obvious reasons).

Devto and REST

The devto API is pretty straight-forward.

"How to choose the right Rust HTTP client" gives a good overview of Rust HTTP crates. In this case, I settled on reqwest.

// Helper state
pub struct DevtoCrossPublish<'s> {
    settings: &'s Settings,
    api_token: String,
    client: reqwest::Client,
}

// Serialized to JSON and used as body of HTTP request
#[derive(serde::Serialize)]
struct Article {
    title: String,
    body_markdown: String,
    published: bool,
    canonical_url: Option<String>,
    #[serde(skip_serializing_if = "Vec::is_empty")]
    tags: Vec<String>,
    series: Option<String>,
}

// API is expecting:
// {
//  article: {
//      title: ...
//      body_markdown: ...
//  }
// }
#[derive(serde::Serialize)]
struct Body {
    article: Article,
}

Posting a new article is pretty simple:

let body = Body {
    article: //...
};
self.client.post("https://dev.to/api/articles")
    // Authenticate
    .header("api-key", &self.api_token)
    .json(&body)
    .send()
    .await

Updating an article takes a bit more. Need to find the devto article corresponding to the original. I decided the canonical URL was the best way to do this because once published it shouldn't change.

// Get page of most recent articles
let me_articles = self.client.get("https://dev.to/api/articles/me")
    .header("api-key", &self.api_token)
    .send()
    .await;

// Find article with matching canonical URL, if it exists
let compare_val = &post.front_matter.canonical_url;
let existing_id = if let Ok(me_articles) = me_articles {
    let me_articles = me_articles.json::<Vec<ArticleResponse>>().await?;
    me_articles.iter()
        .find(|a| self.compare(a, &compare_val))
        .and_then(|x| Some(x.clone()))
} else {
    None
}

// Note `put` and URL for existing article
self.client.put(format!("https://dev.to/api/articles/{}", existing_id))
    .header("api-key", &self.api_token)
    .json(&body)
    .send()
    .await

Medium and REST

The medium API is pretty limited. You can post new articles, but you can't update (i.e. PUT) existing articles. In fact, you can't even GET your articles. Regardless, I'd still like to check for an article using the canonical URL to avoid POSTing a duplicate. Unfortunately, it seems the only viable option to obtain your articles is to get your RSS feed, get individual posts, and check the HTML <head> element for the canonical link.

Based on this article, I decided to try quick-xml. In Cargo.toml:

[dependencies]
quick-xml = { version = "0.22", optional = true }
rss = { version = "1.10", optional = true }

// Get RSS feed
let feed = self.client.get(format!("https://medium.com/feed/@{}", user.username))
    .send()
    .await?
    .bytes()
    .await?;
// Parse RSS feed
let channel = rss::Channel::read_from(&feed[..])?;
// Iterate over links to posts
for item in channel.items {
    if let Some(link) = item.link {
        // Get article
        let story = self.client.get(link)
            .send().await?
            .text().await?;
        // Extract canonical `<link>` from `<head>`
        // ...
    }
}

When you get an article, the response body contains:

<!-- SNIP -->
<head>
    <!-- SNIP -->
    <link data-rh="true" rel="canonical"
            href="https://rendered-obsolete.github.io/2021/05/03/dotnet_calli.html" />
    <link data-rh="true" rel="alternate" href="..." />
    <script data-rh="true" type="application/ld+json">...</script>
    <link rel="preload" href="..." as="script">
    <style type="text/css" data-fela-rehydration="456" data-fela-type="STATIC">
    <!-- SNIP -->
</head>
<!-- SNIP -->

You can see the canonical URL with rel="canonical". In theory we should be able to enable the "serialize" feature of quick-xml and follow the example to decode with:

#[derive(Debug, serde::Deserialize, PartialEq)]
struct Html {
    head: Head,
}

#[derive(Debug, serde::Deserialize, PartialEq)]
struct Head {
    title: String,
    links: Vec<Workaround>,
}

#[derive(Debug, serde::Deserialize, PartialEq)]
#[serde(rename_all = "lowercase")]
enum Workaround {
    Link(Link),
    Script,
}

#[derive(Debug, serde::Deserialize, PartialEq)]
struct Link {
    rel: String,
    href: String,
    sizes: Option<String>,
}

let story: Html = quick_xml::de::from_str(&story)?;

enum Workaround is needed because serde doesn't like the <script> mixed in between <link>s (see this issue). However, it fails with EndEventMismatch { expected: "link", found: "head" }. If you look at the last <link> in the response it's never closed with />- HTML isn't necessarily valid XML. So we'll instead need to follow the reader example and manually traverse the HTML:

let mut reader = quick_xml::Reader::from_str(xml);
reader.trim_text(true);

let mut buf = Vec::new();
let mut is_head = false;
loop {
    use quick_xml::events::{Event, BytesEnd, BytesStart};

    match reader.read_event(&mut buf) {
        Ok(Event::Start(ref e)) if e.name() == b"head" => {
            is_head = true;
        },
        Ok(Event::End(ref e)) if is_head && e.name() == b"head" => {
            is_head = false;
            break;
        },
        Ok(Event::Empty(ref e)) if is_head && e.name() == b"link" => {
            // Check `e.attributes()` key/value for `rel="canonical"`, etc.
        },
        Ok(Event::Eof) => break,
    }
    buf.clear();
}

Posting an article is similar to devto:

let resp = self.client.post(format!("{}/users/{}/posts", URL, user.id))
    .header("Authorization", format!("Bearer {}", self.api_token))
    .json(&body)
    .send()
    .await?;

Hashnode and GraphQL

Hashnode provides a GraphQL API. A GraphQL tutorial is out of scope, but I'll cover the basics. https://graphql.org/learn/ is a good place for further information.

Hashnode has a web-based API sandbox, but the VSCode extension is even comfier because you can test your queries and then use them directly in Rust:

Populate .graphqlrc.yaml with:

schema: https://api.hashnode.com/
extensions:
  endpoints:
    default:
      url: https://api.hashnode.com/
      headers:
        Authorization: ${HASHNODE_API_TOKEN}

The will give requests made by the VSCode extension an Authorization header based on HASHNODE_API_TOKEN environment variable.

Create src/hashnode.graphql:

# `query` keyword and the name we'll use in client code
query Tags {
  # The query we'll run
  tagCategories {
    # Fields to include in response
    _id
    name
    slug
  }
}

Click Execute Query in VSCode and it should output:

{
  "data": {
    "tagCategories": [
      {
        "_id": "56744721958ef13879b94cad",
        "name": "JavaScript",
        "slug": "javascript"
      },
      ...

Note the fields in the response match what we requested.

The main Rust code-first GraphQL project seems to be Juniper. Seeing as how I can't abide code littered with string liter-als (pun intended), the schema-first options boil down to juniper-from-schema and graphql-client. I started with the latter mainly because it has far more stars on github and downloads on crates.io, but also because their examples use reqwest.

To use in Rust:

# Download `graphql-client` tool to `~/.cargo/bin/` (add to $PATH)
cargo install graphql_client_cli
# Get hashnode API schema
graphql-client introspect-schema --output hashnode_schema.json https://api.hashnode.com/

Time for macro magic:

#[derive(graphql_client::GraphQLQuery)]
#[graphql(
    schema_path = "hashnode_schema.json",
    query_path = "src/hashnode.graphql",
    // Added to `#[derive()]` on `ResponseData`
    response_derives = "Debug"
    )]
// Name matches query in hashnode.graphql
pub struct Tags;

let body = Tags::build_query(tags::Variables);
// Post query to hashnode server
let resp = self.client.post("https://api.hashnode.com/")
    .json(&body)
    .send()
    .await?;
// Response is GraphQL `[Tags]` type (array itself and contained items can be null).
// But, `[Tags!]!` (nothing is null) is easier to deal with in Rust
let categories: Vec<tags::TagsTagCategories> = {
    // Make sure to wrap your `ResponseData` with `graphql_client::Response`!
    // If you `resp.json<test::ResponseData>()` you'll get nothing because the 
    // response is `{ data: ResponseData }`
    let categories: graphql_client::Response<tags::ResponseData> = resp.json().await?;
    categories.data
        .and_then(|d| d.tag_categories)
        // Turn `Option<Vec<Option<_>>>` into `Vec<_>`
        .unwrap_or_default()
        .into_iter()
        .flatten()
        .collect()
};

That probably looks like a lot of code and types. Rust can infer many of the types and I could have imported the namespaces, but I find it helpful to see explicit modules and types until you understand what's going on. The last third is just collapsing Option<>/None.

If you want to see what the #[derive()] and #[graphql()] macros are doing behind the scenes, you can explicitly generate Rust source:

graphql-client generate src/hashnode.graphql --output-directory src --schema-path ./hashnode_schema.json

hashnode.rs contains the generated code:

pub struct Tags;
pub mod tags {
    #![allow(dead_code)]
    pub const OPERATION_NAME: &'static str = "Tags";
    pub const QUERY : & 'static str = "query Tags {\n  tagCategories{\n    _id\n    name\n    slug\n  }\n}" ;
    use serde::{Deserialize, Serialize};
    #[allow(dead_code)]
    type Boolean = bool;
    #[allow(dead_code)]
    type Float = f64;
    #[allow(dead_code)]
    type Int = i64;
    #[allow(dead_code)]
    type ID = String;
    #[derive(Deserialize)]
    pub struct TagsTagCategories {
        #[serde(rename = "_id")]
        pub id: ID,
        pub name: String,
        pub slug: String,
    }
    #[derive(Serialize)]
    pub struct Variables;
    #[derive(Deserialize)]
    pub struct ResponseData {
        #[serde(rename = "tagCategories")]
        pub tag_categories: Option<Vec<Option<TagsTagCategories>>>,
    }
}
impl graphql_client::GraphQLQuery for Tags {
    type Variables = tags::Variables;
    type ResponseData = tags::ResponseData;
    fn build_query(variables: Self::Variables) -> ::graphql_client::QueryBody<Self::Variables> {
        graphql_client::QueryBody {
            variables,
            query: tags::QUERY,
            operation_name: tags::OPERATION_NAME,
        }
    }
}

Main take-aways:

You end up using both tags:: and Tags:: (note capital T)
serde is used for serialization to/from JSON
The nullable GraphQL type TYPE becomes Option<TYPE> in Rust, while the non-nullable GraphQL TYPE! becomes TYPE

Posting an article can be done with a "mutation". This example is interesting because it shows how to pass arguments to GraphQL with Variables. In hashnode.graphql add:

# We'll provide `$input` and `$pubId` arguments in Rust
mutation CreatePubStory($input: CreateStoryInput!, $pubId: String!, $hideFromFeed: Boolean = false) {
  # This is the mutation, we use arguments from above
  createPublicationStory(input: $input, publicationId: $pubId, hideFromHashnodeFeed: $hideFromFeed) {
    # Response fields specified in fragment
    ...createResponseFields
  }
}

# Fragment usable on CreatePostOutput types
fragment createResponseFields on CreatePostOutput {
  code
  success
  message
}

And the corresponding Rust is similar to before:

#[derive(graphql_client::GraphQLQuery)]
#[graphql(
    schema_path = "hashnode_schema.json",
    query_path = "src/hashnode.graphql",
    response_derives = "Debug",
    // Add `#[derive(Default)]` to `Variables` type
    variables_derives = "Default"
    )]
pub struct CreatePubStory;

// Struct used as mutation input
let input = create_pub_story::CreateStoryInput {
    content_markdown: post.body,
    tags, // Tags from earlier query
    title: post.front_matter.title,
    // etc...
};
// Pass `$input` and `$pub_id` arguments to `CreatePubStory` mutation in hashnode.graphql via `Variables` struct
let body = CreatePubStory::build_query(create_pub_story::Variables {
    input,
    pub_id: self.pub_id.clone(),
    ..Default::default() // Since we added #[derive(Default)]
});

let resp = self.client.post("https://api.hashnode.com/")
    // Required authorization header
    .header("Authorization", &self.api_token)
    .json(&body)
    .send()
    .await?;

let resp: graphql_client::Response<create_pub_story::ResponseData> = resp.json().await?;

Updating a post can be accomplished with updateStory() mutation. However, hashnode's API doesn't seem to provide a way to retrieve a post's canonical URL, so we can't use that to locate an existing article like with devto. Instead, I'll generate a slug from the filename and look for that. That means I'll have to handle renaming a file, but I figure that's less likely to happen than changing the title, body, or something else.

Miscellany

To close this out, a few smaller topics that were useful.

Async

Tokio seems to be the de-facto solution for async Rust. Many of the required types like Future are in the standard library, but the futures crate is still pretty useful. In Cargo.toml:

[dependencies]
futures = "0.3"
tokio = { version = "1.5.0", features = ["full"] }

In main.rs:

#[tokio::main]
async fn main() -> Result<()> {
    start().await
}

Elsewhere:

let mut futures: Vec<futures::future::LocalBoxFuture<Result<()>>> = vec![];
futures.push(Box::pin(async move {
    let devto = devto::DevtoCrossPublish::new(token.clone(), &settings);
    devto.publish(post).await
}));
// Do the same for medium and hashnode...

let results = futures::future::join_all(futures).await;

join_all is similar to the join! macro and essentially awaits an iterator of futures.

Error Handling

There's been several chapters in the Rust error handling story; failure, error-chain, and a few others I'm forgetting. Apparently those are old and busted, anyhow and thiserror are new hotness.

In Cargo.toml:

[dependencies]
anyhow = "1.0"
thiserror = "1.0"

anyhow works with the std::error::Error trait, propogates errors with ?, allows specifying additional context, and has some useful macros:

use anyhow::Context;

fn can_fail() -> anyhow::Result<()> {
    this_fails().context("Failed to whatever")?;
    also_fails().with_context(|| format!("Check this: {}", important_thing))?;

    if whatever {
        // anyhow! macro produces a anyhow::Error
        return Err(anyhow!("Missing attribute: {}", missing));
    } else if other_thing {
        // bail! macro is shorthand for above
        bail!("Missing attribute: {}", missing);
    }
}

From the docs:

(anyhow::)Error works a lot like Box<dyn std::error::Error>, but with these differences:

Error requires that the error is Send, Sync, and 'static.

Error guarantees that a backtrace is available, even if the underlying error type does not provide one.

Error is represented as a narrow pointer — exactly one word in size instead of two.

thiserror is complementary by helping create custom error types that impl std::error::Error and impl Display:

#[derive(thiserror::Error, Clone, Debug)]
pub enum Error {
    #[error("Bad path, expected {expected}: {found}")]
    BadPath {
        expected: String,
        found: std::path::PathBuf,
    },
    //...
}

fn function() -> anyhow::Result<()> {
    option.ok_or(Error::BadPath { expected: "...".to_owned(), found: "...".to_owned() })?;
}

It has several other features, check the docs for more information.

If you get this error:

error[E0308]: mismatched types
   --> src/platforms/hashnode.rs:148:13
    |
148 |         Err(Error::NotFound)
    |             ^^^^^^^^^^^^^^^^ expected struct `anyhow::Error`, found enum `Error`

Use impl From:

Err(Error::NotFound.into())

Logging

I'd wager most people are familiar with log and a logger implementation like env_logger:

use log::{info, trace, warn};
env_logger::init();
info!("Information: {}", value);

tracing is similar. In Cargo.toml:

[dependencies]
tracing = "0.1"
tracing-subscriber = "0.2"

use tracing::{debug, error, info, trace, warn};
tracing_subscriber::fmt().init();

You can use RUST_LOG to filter output:

# PowerShell
$env:RUST_LOG="warn,cargo_bullhorn=trace"
# bash-like
export RUST_LOG=warn,cargo_bullhorn=trace

Tracing provides additional features: logging sections of code with RAII spans, decorator for functions, and more. Check the docs.

Please stop the forem app pop-up

jeikabu — Sun, 30 May 2021 06:41:29 +0000

Please stop the forem app message from constantly popping up on mobile. It's annoying.

Benchmarking in .NET or How I Learned to Stop Worrying and Love Perf Regression Testing

jeikabu — Sat, 22 May 2021 00:00:00 +0000

In an earlier post we were looking at the performance of function pointers introduced in .NET 5/C# 9.0. To do that we used BenchmarkDotNet (docs), the de-facto .NET benchmarking framework. This is a shotgun dump of notes to get you writing your own benchmarks- mainly for performance regression testing.

Basics

# Install benchmark template
dotnet new -i BenchmarkDotNet.Templates
# Create new benchmark project `benchmarks`
dotnet new benchmark -o benchmarks --console-app -v 0.12.1

See BenchmarkDotNet template options

Confusingly, you’ll find references to installing BenchmarkDotNet.Tool global tool which has been deprecated (see issue 1670 and PR 1572). Likewise, the dotnet project template BenchmarkDotNet.Templates defaults to: NET Core App 3.0 (which is out of support), and version 0.12.0 of BenchmarkDotNet (which doesn’t support NET 5.0 and may fail with System.NotSupportedException: Unknown .NET Runtime). Open benchmarks/benchmarks.csproj and replace:

-<TargetFramework>netcoreapp3.0</TargetFramework>
+<TargetFramework>net5.0</TargetFramework>

The template is pretty simple, so it’s just as easy to run dotnet new classlib -o benchmarks and populate benchmarks.csproj with:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <OutputType>Exe</OutputType>
  </PropertyGroup>
  <PropertyGroup>
    <PlatformTarget>AnyCPU</PlatformTarget>
    <DebugType>portable</DebugType>
    <DebugSymbols>true</DebugSymbols>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <Optimize>true</Optimize>
    <Configuration>Release</Configuration>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.12.1" />
    <PackageReference Include="BenchmarkDotNet.Diagnostics.Windows" Version="0.12.1" Condition="'$(OS)' == 'Windows_NT'" />
  </ItemGroup>
</Project>

In benchmarks/Program.cs, replace the main body with the following to get the same functionality as the benchmark runner CLI (from the docs):

using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Running;

namespace benchmarks
{
    class Program
    {
        static void Main(string[] args) =>
            BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
    }
}

Create a benchmark in Benchmarks.cs:

using BenchmarkDotNet;
using BenchmarkDotNet.Attributes;

namespace benchmarks
{
    public class Delegate
    {
        [Benchmark]
        public void CallDelegate()
        {
            // Do nothing for now
        }
    }
}

Try running the benchmarks:

dotnet run --project ./Benchmarks/benchmarks.csproj -- --filter "benchmarks.*"

If you leave off --filter it will wait for input from stdin.

To see the list of CLI options:

dotnet run --project ./Benchmarks/benchmarks.csproj -- -h

Invariably, you’re testing some other library as opposed to writing stand-alone benchmarks. And then at some point it may fail with Assembly XXX which defines benchmarks references non-optimized YYY. Everything needs to be compiled for release:

# Add a project reference to assembly we're benchmarking
dotnet add benchmarks reference ./nng.NET.Shared/nng.NET.Shared.csproj
# Note --configuration
dotnet run --project ./Benchmarks/benchmarks.csproj --configuration release -- --filter "benchmarks.*"

Similar to other testing frameworks, you can have initialization and cleanup (that is excluded from benchmark timings) with [GlobalSetup]/[GlobalCleanup] and [IterationSetup]/[IterationCleanup] for per-class and per-iteration behavior, respectively (docs):

public class Delegate
{
    delegate int nng_aio_set_output_delegate(nng_aio aio, int index, IntPtr arg);
    nng_aio_set_output_delegate nng_aio_set_output;

    [GlobalSetup]
    public void GlobalSetup()
    {
        var handle = DllImport.Load();
        var ptr = NativeLibrary.GetExport(handle, "nng_aio_set_output");
        nng_aio_set_output = Marshal.GetDelegateForFunctionPointer<nng_aio_set_output_delegate>(ptr);
    }

    [Benchmark]
    public void CallDelegate()
    {
        var _ = nng_aio_set_output(nng_aio.Null, 9, IntPtr.Zero);
    }
}

There are two main approaches to defining benchmarks:

Configs (in either fluent or object style)
Class/function attributes

Which allows you to configure all sorts of interesting things: benchmark parameters, runtime jobs (e.g. .NET version, JIT/GC), etc.

If you’ve got multiple classes the CLI results are spread out. To get a single, tidy table at the end:

var config = ManualConfig.Create(DefaultConfig.Instance)
    .WithOptions(ConfigOptions.JoinSummary);
BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args, config);

ManualConfig has numerous more options. Everything in ConfigOptions can be specified as CLI arguments after --. For example, JoinSummary corresponds to --join. Reference -- -h for others.

To quickly iterate at the expense of accuracy use: --job Dry, --runOncePerIteration, or some combination of --warmupCount 1 --iterationCount 1 --invocationCount 1 --unrollFactor 1.

Reports

By default, csv, html, and markdown results are written to BenchmarkDotNet.Artifacts/results/. Other formats are possible with -e, and the base output path can be specified with -a/--artifacts. The markdown is suitable for posts such as this, CI web interfaces, etc.:

BenchmarkDotNet=v0.12.1, OS=macOS 11.3.1 (20E241) [Darwin 20.4.0]
Intel Core i5-8279U CPU 2.40GHz (Coffee Lake), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.101
[Host] : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT
DefaultJob : .NET Core 5.0.1 (CoreCLR 5.0.120.57516, CoreFX 5.0.120.57516), X64 RyuJIT

Type	Method	Mean	Error	StdDev	Ratio	RatioSD
Delegate	CallDelegate	16.934 ns	0.1687 ns	0.1578 ns	16.23	0.23
DllImport	CallDllImport	6.713 ns	0.0666 ns	0.0590 ns	6.44	0.09
Interface	CallInterfaceToDllImport	9.884 ns	0.1892 ns	0.2652 ns	9.55	0.35
Pointer	CallFunctionPointer	6.764 ns	0.1519 ns	0.1347 ns	6.49	0.18
DllImport	CallManaged	1.042 ns	0.0125 ns	0.0110 ns	1.00	0.00

To get graphs, install the R programming language and use -e rplot. If rscript is in your PATH environment variable, it will automatically output several png files, otherwise you can manually run rscript BenchmarkDotNet.Artifacts/results/BuildPlots.R:

Comparisons

What’s “fast” (or “slow”)? Benchmarks are sometimes useless without something to compare them to.

You can define a baseline:

[Benchmark(Baseline = true)]
public void CallManaged()
{
    var _ = Managed(IntPtr.Zero, 9, IntPtr.Zero);
}

// Prevent inline-ing
[MethodImpl(MethodImplOptions.NoInlining)]
static nint Managed(nint _unused0, int _unused1, nint _unused2) => 0;

Having a baseline also gives you “ratio” and “ratioSD” (StdDev) columns in the output.

BenchmarkDotnet doesn’t natively support comparing results from different runs (i.e. before and after code changes), but you can use ResultsComparer from the .NET project:

# Clone repo and build ResultsComparer
git clone https://github.com/dotnet/performance dotnet_perf
pushd dotnet_perf/src/tools/ResultsComparer
dotnet build -c release
popd
# View CLI options
dotnet ./dotnet_perf/artifacts/bin/ResultsComparer/Release/net5.0/ResultsComparer.dll -h

# Run benchmarks with full JSON reports
dotnet run --project ./Benchmarks/benchmarks.csproj -c release -- -f "benchmarks.*" -e fulljson
# Compare with results in ./base
dotnet ./dotnet_perf/artifacts/bin/ResultsComparer/Release/net5.0/ResultsComparer.dll --base ./base --diff ./BenchmarkDotNet.Artifacts --threshold '5%'

ResultsComparer requires “full” JSON reports. So, you’ll have to do one of:

Run with -e fulljson
Call AddExporter(BenchmarkDotNet.Exporters.Json.JsonExporter.Full) on a config
Apply [BenchmarkDotNet.Attributes.JsonExporterAttribute.Full] attribute

--threshold sets the permitted variation between results. It can be a percentage, or length of time like 1ms.

Stdout contains markdown results:

Slower	diff/base	Base Median (ns)	Diff Median (ns)	Modality
benchmarks.Interface.CallInterfaceToDllImport	112104.10	11.05	1239001.46

Oops, looks like my Thread.Sleep(1) had a measureable impact.

ResultsComparer has some unexpected behavior. It throws an exception when there’s multiple reports in the same folder, so you may need to clear results from repeated runs. And it doesn’t work with results from “dry” jobs.

CI

We’re looking to add this as part of our Github Action workflows, but it applies equally well to any other CI/CD platform.

It’s tempting to fail a build/PR if any benchmark is slower. But, sometimes it’s unavoidable with a bug fix or “correct” implementation. IF we wanted to do it anyway, check if any test is “Slower”:

# Output comparison as CSV `./csv`
dotnet ResultsComparer.dll --base ./base --diff ./BenchmarkDotNet.Artifacts --threshold '5%' --csv ./csv
# Parse csv, give it a header (it doesn't have one), and get 3rd column
$column = Import-csv -Path ./csv -Delimiter ';' -Header 'Benchmark','Source','Result' `
    | Select-Object 'Result'
# Check for "Slower"
($column | % { $_.Result }).Contains("Slower")

What we’d really like is a summary in the comments of a pull request- similar to our code coverage reports. There’s multiple ways to add a comment from a workflow:

github-script block
curl/REST
Community marketplace action (option1, option2)

Using HTTP REST, PowerShell works on all platforms and requires no external dependencies:

# Run ResultsComparer and capture markdown output
$output = dotnet ResultsComparer.dll --base ./BenchmarkDotNet.Artifacts/ --diff ./diff --threshold '5%'
# Convert string[] output to single string with line-breaks
$body = $output -join '`n'
$secure_token = ConvertTo-SecureString -String $GITHUB_TOKEN -AsPlainText
# POST comment to PR
Invoke-WebRequest `
    -Method Post `
    -Headers @{accept= 'application/vnd.github.v3+json'} `
    -Uri https://api.github.com/repos/$USER/$REPO/issues/$PR_ID/comments `
    -Body (ConvertTo-Json @{body=$body}) `
    -Authentication OAuth -Token $secure_token

Where $PR_ID can be obtained from $ in the workflow.

Assuming everything is correct:

We’ve now got a bunch pieces, but haven’t really settled on how we want the process to work. Similar to before, we can attach any of the generated reports with upload-artifact/download-artifact or upload-release-asset as build artifacts or release assets, respectively. And then generate comparison reports. But, it feels best to use a wide gamut of fine-grained benchmarks to catch performance regressions in PRs, and then a smaller number of broader, generalized tests to highlight changes between releases. This calls for more experimentation.

Physical Security

jeikabu — Sat, 15 May 2021 00:00:00 +0000

I love a good caper film. When I was a wee lad my father introduced me to The Great Train Robbery. Over the years I’ve seen several memorable others Rififi, A World Without Thieves, several by David Mamet like The Spanish Prisoner. Just the other day I watched Inside Man.

Numerous vulnerabilities like Spectre, Man-in-the-middle, SQL injection, keyloggers, phishing and a legion of others have loomed over the IT sphere for decades. Seems like every month there’s a new, “largest ever”, high profile security incident.

The never ending struggle has given rise to a bevy of best-practices and technologies: asymmetric cryptography, penetration testing, salting hashed passwords, numerous VM enhancements (like SEV), two-factor authentication and countless others.

Locks and Lock Picking

Many people working in IT are accustomed to the idea of a (climate-controlled) server room with limited access. Anyone that’s needed to restore root/administrator access (and perhaps keeps a bootable USB drive of associated tools on their person) understands the fundamental risk.

In most cases, these are protected by locks controlled via keypad, access card, or biometrics. But the pre-cursor to them, the humble pin tumbler lock, is still found in a variety of settings like doors and cabinets that likewise protect equipment.

About a year ago I became curious about lock picking. I can’t remember which movie I was watching, but someone is locked in a basement and manages to remove their restraints and escape- seemed like a useful skill to have.

Numerous resources exist for budding locksmiths such as The Complete Book of Locks and Locksmithing. It covers the design of several types of locks, how to duplicate keys and design a master-key system, and lockpicking among other topics relevant in the trade. Additionally, there are an assortment of educational tools such as practice locks:

After which you can graduate to over-the-counter locks available in any hardware store:

Most surprising to me was that even with inexpensive, entry-level picks, anything less than “high security” locks take little practice. NB : possession of lockpicks by unlicensed individuals may not be legal where you are.

Continuing with my cinematic theme, The Next Three Days features Russel Crowe’s failed attempt to use a “bump key”. A variety of lockpicking using a specially cut key for particular locks.

Does Physical Security Really Exist?

Over the years, researchers have demonstrated borderline outlandish, largely theoretical attacks that don’t even require physical access. For example, using hard-drive acoustics, screen brightness, and several others.

And more recently, the article that inspired this post, Researchers Can Duplicate Keys from the Sounds They Make in Locks.

Native Code in .NET 5.0 and C# 9.0

jeikabu — Mon, 03 May 2021 00:00:00 +0000

A while back we covered working with native assemblies. It's worth re-reading to familiarize yourself with the problem and prior solutions. There's been a few new options introduced in .NET Core and .NET 5.0.

NativeLibrary

The NativeLibrary static class was introduced in .NET Core 3.0 and makes it easy to use DllImport in a portable way. To call a function nng_alloc() in a native shared library:

// In `nng.NETCore` managed assembly
namespace nng.Native.Basic
{
    public sealed class UnsafeNativeMethods
    {
        public const string NngDll = "nng";

        [DllImport(NngDll, CallingConvention = CallingConvention.Cdecl)]
        public static extern IntPtr nng_alloc(UIntPtr size);
    }
}

// In assembly that references `nng.NETCore`
namespace benchmarks
{
    class Program
    {
        static void Main(string[] args) 
        {
            NativeLibrary.SetDllImportResolver(typeof(nng.Native.Basic.UnsafeNativeMethods).Assembly, DllImportResolver);
            // Calling DllImport'd method triggers resolver
            nng.Native.Basic.UnsafeNativeMethods.nng_alloc(UIntPtr.Zero);
        }

        static IntPtr DllImportResolver(string libraryName, System.Reflection.Assembly assembly, DllImportSearchPath? searchPath)
        {
            // Load platform-specific unmanaged shared library. e.g.:
            var path = "runtimes/osx-x64/native/libnng.dylib";
            return NativeLibrary.Load(path);
        }
    }
}

None of the mess of calling dlopen() on Mac/*nix or LoadLibrary() on Windows to load a *.so or *.dll, respectively.

If DYLD_PRINT_LIBRARIES env variable is set before running the code:

dyld: loaded: <264EA187-4189-3CE5-82C3-9746FFE68B66> runtimes/osx-x64/native/libnng.dylib

Function Pointers

.NET 5.0 and C# 9.0 introduced function pointers as "a performant way to call native functions from C#".

public class Pointer
{
    public void CallFunctionPointer()
    {
        // Load platform-specific native library
        var handle = NativeLibrary.Load("xxx");
        // Get address of exported symbol
        var ptr = NativeLibrary.GetExport(handle, "nng_alloc");
        // Function pointers require `unsafe` context
        unsafe
        {
            var nng_alloc = (delegate* unmanaged[Cdecl]<nuint, nint>)ptr;
            // Call through function pointer
            nng_alloc(0);
        }
    }
}

NativeLibrary.GetExport() returns the address of a symbol exported by a shared library. It effectively replaces calling the native functions GetProcAddress() and dlsym() on Windows and Mac/*nix, respectively.

Function pointers use the funky syntax delegate* unmanaged[Cdecl]<args, retval> (detailed here):

Part	Description
`delegate*`	Reuses `delegate` keyword and `*` denotes an unsafe pointer
`unmanaged`	Either `managed` or `unmanaged` depending on the function
`[Cdecl]`	Optional after `managed`. One or more comma-separated `System.Runtime.CompilerServices.CallConv*` values. Here `CallConvCdecl`
`<args, retval>`	Function signature types- arguments followed by return value (same as `System.Func`)

Comparison

A post on medium compared the performance of different interop approaches a few years ago. We can re-run the same comparison passing arguments by-value with .NET 5.0 using BenchmarkDotNet.

nng_aio_set_output(null, 9, null) should be a reasonable choice to benchmark native interop performance because for values greater than 3 it immediately returns EINVAL.

The results on a 2019 13" MacBook Pro with Quad-core i5 @ 2.4 GHz, 16 GB RAM, running macOS Big Sur (11.3), and .NET 5.0.101:

Method	Mean	Error	StdDev	Ratio	RatioSD
CallDelegate	16.989 ns	0.3568 ns	0.3163 ns	13.01	0.74
CallDllImport	7.184 ns	0.1639 ns	0.1279 ns	5.49	0.22
CallInterfaceToDllImport	10.939 ns	0.3972 ns	1.1711 ns	8.07	0.75
CallFunctionPointer	6.733 ns	0.1613 ns	0.2208 ns	5.22	0.30
CallManaged	1.305 ns	0.0548 ns	0.0563 ns	1.00	0.00

Looking at CallDllImport and CallFunctionPointer, function pointers are ~5-10% faster than DllImport.

CallDelegate uses Marshal.GetFunctionPointerForDelegate to call via a managed delegate. Similar to before, it's slowest by a sizeable margin.

CallInterfaceToDllImport is the way we're currently calling native functions in nng.NET; through an interface to static functions decorated with DllImport. Based on these results, leveraging NativeLibrary and function pointers we can reduce interop overhead by ~33%. Not too shabby.

To see what's going on, we can use ILSpy via the VSCode plugin to decompile the assembly. Open the Command Palette (CMD+SHIFT+E), run "ILSpy: Decompile IL Assembly (pick file)", and select the desired assembly/*.dll. In the Explorer View (CMD+SHIFT+E), browse to ILSPY DECOMPILED MEMBERS > benchmarks > Pointer > CallFunctionPointer:

    IL_0000: ldarg.0
    IL_0001: ldfld method int32 *(native int, uint32, native int) benchmarks.Pointer::nng_aio_set_output /* 04000003 */
    IL_0006: stloc.1
    IL_0007: ldsfld native int [System.Runtime]System.IntPtr::Zero /* 0A000015 */
    IL_000c: ldc.i4.s 9
    IL_000e: ldsfld native int [System.Runtime]System.IntPtr::Zero /* 0A000015 */
    IL_0013: ldloc.1
    IL_0014: calli unmanaged cdecl int32(native int, uint32, native int) /* 11000004 */
    IL_0019: stloc.0
    IL_001a: ret

And the decompiled source:

public unsafe void CallFunctionPointer()
{
    IntPtr intPtr = nng_aio_set_output;
    int num = ((delegate* cdecl<IntPtr, uint, IntPtr, int>)intPtr)(IntPtr.Zero, 9u, IntPtr.Zero);
}

So, for whatever reason it turns our function pointer into an IntPtr and then casts it to cdecl (which must be an internal symbol because we can't use it directly). But the key thing is the use of calli. Compare that with benchmarks > DllImport > CallDllImport:

    IL_0000: ldsfld valuetype [nng.NET.Shared]nng.Native.nng_aio [nng.NET.Shared]nng.Native.nng_aio::Null /* 0A000014 */
    IL_0005: ldc.i4.s 9
    IL_0007: ldsfld native int [System.Runtime]System.IntPtr::Zero /* 0A000015 */
    IL_000c: call int32 [nng.NET]nng.Native.Aio.UnsafeNativeMethods::nng_aio_set_output(valuetype [nng.NET.Shared]nng.Native.nng_aio, uint32, native int) /* 0A00001C */
    IL_0011: stloc.0
    IL_0012: ret

DllImport uses call. Based on the docs for call and calli opcodes, call uses a "method descriptor" that is a "metadata token" (and works with virtual functions) while calli is just "a pointer to an entry point"- and apparently faster.

Additional reading:

3D Printing a Custom Device Case

jeikabu — Fri, 30 Apr 2021 00:00:00 +0000

I got a 3D printer around 2 years ago and other than the occasional dodad, it hasn’t been used for the intended purpose of printing custom cases for projects (like my solar-powered USB hardware). I finally got around to investigating this because I need a case for my CAPA13R mini-PC.

There’s already tons of resources on the web to help you pick a 3D printer, explain the different printing filament materials, tutorials on modeling and slicing, and so on. I’m just going to cover some tips to start making things with fairly precise constraints (but no moving parts).

Creating 3D Models

Despite being in the game industry 15 years and working along-side people who use DCC tools like 3ds Max, Maya, and ZBrush everyday, I possess no 3D-modeling skills (other than briefly using MilkShape 3D in uni). That gap needs to be closed first.

I decided on the following 3D modeling software requirements:

Precise dimensions to achieve a good fit
Free (preferably FOSS)
Offline (not web/cloud-based)
Multi-platform (at least Mac and Windows)

The parametric modeler FreeCAD meets all these requirements so I’ve been using that, but there’s other options that might work just as well. FreeCAD is a non-trivial program so I sought out some tutorials. There’s an abundance of material online, “DrVax” has a few videos on FreeCAD specific to 3D printing.

Blender might also be an option and better suited to general 3D modeling, but I didn’t investigate how easy it is to use for CAD design. OpenSCAD looks particularly interesting.

The basic process to 3D print any object is:

Measure the various dimensions for the 3D object
Create 3D model and export as STL (or another format supported by slicer)
Use slicer software to generate g-code for 3D printer
Print

I’m using the slicer developed for my printer, but you can search for what’s recommended for your printer or to get more advanced features.

Modeling/FreeCAD

Export a simple object from the modeler and import into your slicer to figure out the position/orientation of the print bed relative to your modeling (probably XY-plane and +Z axis). You can always move things around, but this makes it easier to think about.
Start with a simple object and work on increasingly complex objects to get a handle on the modeling software, slicer, and 3D printer.
Use a meaningful reference point (e.g. the corner of a real-world physical object) as the origin from which to measure and model. This keeps dimensions/orientation simple and avoids additive in-accuracy/imprecision.
In FreeCAD, Ctrl+Shift rotates and Ctrl+Cmd pans the view with the left button (i.e. when using a trackpad).
Model in a natural, forward order as if working with clay; model and validate major/important features first and leave details and flourishes (i.e. fillet and chamfer) until the end. This minimises re-work when adjustments need to be made.
FreeCAD’s model view is like snapshots back in time with pointers into the past as you make changes. Don’t alter the space-time continuum lest you create dangling pointers! At least until you learn to work with it.
Keep it data-driven (FreeCAD’s spreadsheet workbench and cell aliases); =board_width + X is more meaningful than 15.2 and makes it easier to tweak related dimensions.
To align geometry where there’s no useful reference, create a line to snap things to and toggle construction mode:
Rename components (i.e. sketches/pads/pockets) so as the model becomes more complex you can find things.
Superfluous air-vents or similar details help reduce the amount of material, print time, and overall weight:

Consider dimensions where tolerances are permitted, note where wires/connectors go, account for how an object would be inserted into the print and leave extra room. Similarly, pad accordingly (e.g. away from measured dimensions).
Model the bare-minimum (reduced pad dimension in FreeCAD) and print a prototype to test fitting.
If you’re going to print multiple pieces separately, create them as multiple bodies in FreeCAD and let the sketcher make clones or use shape binder to create relationships:
Exaggerate features that you think only need to be a few millimetres. They’re more likely to print correctly, and will be less fragile.

Slicing and Printing

Start with PLA filament as it’s generally the most forgiving and easiest to work with (and often cheapest).
Prototype with a “junk” color you’re unlikely to use for finished products to conserve more useful filaments.
Use slicer low-quality settings to speed up prototyping.
Make sure to check what the slicer is generating- hollow walls use far less material:

Since models print from the bottom upwards, you may need supports for overhangs. Most slicers can add these, if that’s not workable you can build them into the model.
Consider whether supports will be easy to remove without damaging the model. Print a small section if you’re not sure.
I tend to set-and-forget models I’ve downloaded, but you’ll want to keep a closer eye on “unproven” models to ensure the printer doesn’t waste a lot of filament making a mess.
If warping occurs, try increasing the platform temperature and/or have the slicer generate a “raft” or “brim” (also see here and here).
Print near an open window or in a well-ventilated area. Even if non-toxic, the smell of molten plastic leaves a lot to be desired.
Use a thin, flat object with a sharp edge (like a paint scraper or knife) to help remove objects from the print bed after everything cools. There’s numerous other tricks.
If you need to print multiple objects with the same material, use the slicer to layout the various models (or duplicates) and do as many as possible in one printing.
If you’re going to print a number of different objects, plan ahead to batch the same material together to minimize the number of times you have to switch filaments (taking into account if your printer has multiple extruders/spools).

Optional (But Often Useful)

Dremel/sand-paper/file(s), thin wire-cutters/X-acto knife, needle-nose pliers, etc.: to help remove supports and weird edges, or make minor alterations to potentially salvage an object with trivial flaws. This might not have the best results, YMMV.
Calipers (digital or analog): makes it easy to take precise measurements for a snug fit or that would be awkward with a ruler:

Small round objects	Precise distances with obstructions

Check thingiverse for “universal” spool holders or other accessories that might help you wrangle filament.
Trashcan: Seriously. Little whisps of plastic, supports, rafts, prototypes, botched prints, etc. quickly litter your work area.
Small hand-held vacuum cleaner and/or something like an anti-static mat to make cleanup a breeze.
Mouse: most modeling software and slicers make good use of a third button.

Final Product

I’m pretty happy with the end results. The CAPA13R board and power supply rest securely in the case, and it looks decent enough to sit in my living room.

Front	Back

Contemplating a second printing to correct a few minor gripes. I printed the top with a brim instead of a raft and the right side warped a tiny bit and is a little ill-fitting, so the top doesn’t snap on quite as well as I envisioned. Also, I printed the top upside down to avoid using supports, but the surface towards the platform/raft doesn’t look as nice.

Rust in Android OS and Linux

jeikabu — Thu, 22 Apr 2021 00:00:00 +0000

Google had two recent announcements of particular interest to Rust enthusiasts: ongoing efforts to introduce Rust into Android OS and the Linux kernel.

While a number of memory-safe languages like Java/Kotlin/go/et al. are used in user-space, the majority of OSes are written in C/C++ where incorrect memory usage is still a source of “high severity” bugs. The idea is that using Rust would avoid such classes of bugs and provide other benefits.

The Linux article has a few specific examples for a simple character device, ioctl handling, and synchronisation primitives. They illustrate some of Rust’s advantages: immutability, static typing, lifetime checks, bounds checks, RAII, required initialisation and error-handling, and so on. Most achieved at compile-time rather than run-time.

Why not bring C++ to Linux? Linus Torvalds had this to say:

LOL. C++ solves none of the C issues, and only makes things worse. It really is a crap language.

Building an AMD Ryzen APU (mini) Game Machine

jeikabu — Tue, 13 Apr 2021 00:00:00 +0000

Between my love of game consoles and working on Z+ with AMD I perhaps have an unhealthy obsession with AMD APUs. When AxiomTek announced (1, 2) a single-board-computer with Zen CPU cores and Vega graphics I was… intrigued, and thought it might make for a nice Steam Box.

Hardware

Details on the CAPA13R board are available from the product page and more detailed spec sheet. Common features:

AMD Ryzen Embedded V1605B or V1807B APU
DDR4 SO-DIMM (up to 16GB of memory)
1 x M.2 Key B (for SSD), 1 x M.2 Key E
2 x 10/100/1000 Mbps Ethernet
2 x USB 3.2 Gen2
2 x HDMI, 1 x DisplayPort++, 1 x LVDS

Notably, it lacks wifi and bluetooth. The wifi I can do without (the board requires external power so an ethernet cable is fine), but I added a USB bluetooth adapter I had laying around.

	V1605B	V1807B
Memory	DDR4-2400	DDR4-3200
Power min. RMS (W)	12V @ 2.5A	12V @ 4.2A

Details on the AMD SoC are available from the Ryzen Embedded V1000 series product page and the product brief. All models have 4 cores (8 threads, 4M L2 cache) Zen micro-architecture CPU married with a Radeon Vega 8 GPU (GCN5):

	V1605B	V1807B
CPU base/max GHz	2.0 / 3.6	3.35 / 3.8
GPU CU/GHz/TFLOPS	8 / 1.1 / 1.1	11 / 1.3 / 1.8
TDP range/nominal (W)	12-25 / 15	35-54 / 45
DDR4 (MT/s / GB/s)	2400 / 19.2	3200 / 25.6

Like NVidia’s Jetson Nano, the memory bandwidth is rather low. For comparison, the original Xbox One/Xbox One S has 8 Jaguar cores (pre-Zen, 8 threads), 12 CUs (GCN2, 1.3/1.4 TFLOPS), and DD3 at 68.3 GB/s. It’s difficult to make an “apples to apples” comparison because Zen has 52% improved IPC, GCN 2 and 5 are fairly distant, and the Xbox has an additional 32 MB of ESRAM at 200+ GB/s (generally used for render targets). But, the V1605V and V1807B are “in the ballpark” of the original Xbox and Xbox S, respectively- memory bandwidth aside.

I picked up a lower-end CAPA13R with the V1605B and added:

Transcend 240GB SATA III 6Gb / s MTS420S 42mm M.2 SSD
HyperX Impact DDR4 16GB, 2666MHz CL15 SODIMM XMP - HX426S15IB2 / 16
PicoPSU-60 12V DC-DC ATX Mini-ITX 0-60W power supply power
Salcar 60 W Transformer Power Adapter (12 V 6 A, 5.5x2.5mm)

Top	Bottom

Until ATX12VO (also see this guide) becomes more of a thing, I’m using a 60W PicoPSU with pins 13 and 14 (OR pin 14- PS-ON- to one of the other GNDs) shorted to power the system. Depending on your power supply of choice, another approach may be necessary.

It’s worth mentioning SSW1 , the small switch to the far left of the external ports. With 1-2 closed (switch away from the external ports) auto power on is disabled and the soft power button is needed to power on the system. With 2-3 closed (switch towards external ports) power on is enabled and just plugging the system in will auto power on.

Benchmarks

Installed Windows 10 20H2/19042.631 and the drivers to get audio and graphics working. Otherwise stuck with the stock install and ran a few benchmarks:

Benchmark	Settings	Score Min/Avg/Max FPS
Unigine: Heaven 4.0	4K/OpenGL	45 1.1 1.8 4.0
Unigine: Heaven 4.0	1920x1200/OpenGL	252 4.5 10.0 24.1
Unigine: Heaven 4.0	1080p/OpenGL	317 5.1 12.6 35.1
Unigine: Superposition v1.1	1080p med/OpenGL	857 5.72 6.41 7.89
Unigine: Valley 1.0	1920x1200/DX11	596 9.7 14.2 24.1

Benchmark	Settings	Score Graphics CPU
3DMark Demo Time Spy v1.2	4K/64-bit	595 529 2085
3DMark Demo Time Spy v1.2	1920x1200/64-bit	641 571 2170

So, not cut out for 4K gaming. However, during none of the benchmarks did the heatsink even become warm, and even the 1080p performance isn’t very good. I wondered if either the PicoPSU or power adapter just wasn’t delivering enough power, but the GPU seemed to be fully engaged:

What stands out is the GPU has only 1 GB of dedicated memory. This was a limiting factor for us with the Z+ console (which had a carve-out of 2GB for dedicated graphics memory leaving 6 GB of system memory). In our case, we had a BIOS option to specify the amount reserved for VRAM, but the CAPA13R BIOS currently has no such option.

In hind-sight I probably should have gone with the V1807V since the GPU is ~60% more powerful.

Software

The default boot prompt timeout is 1s, so you’ll have to be quick to change BIOS settings. Press esc, set Advanced > Smart Fan Function to Enabled. This allows the fan speed to adjust based on the SoC temperature and makes the system much quieter.

Optionally, set Boot > Quiet Boot to Enabled to suppress the BIOS startup screen. However this replaces the Windows 10 logo with the American Megatrends logo- which doesn’t look great. Changing the UEFI logo seems to require a tool like “HackBGRT”, but I decided to not bother because the boot prompt timeout is short enough that my TV often isn’t even on when the BIOS startup is displayed.

Have Steam auto-start in Big Picture Mode, via Steam > Settings :

Configure users to auto-login:

Change registry:
1. Enter Win+R and type regedit OR hit Win key and start typing registry editor, then hit enter
2. Browse to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\PasswordLess\Device change DevicePasswordLessBuildVersion to 0
Enter Win+R and type netplwiz, then hit enter OR click OK
Uncheck Users must enter a user name and password to use this computer and click OK

Now when you restart the system it should login and launch straight into Steam BPM:

With an added USB bluetooth adapter, pair with a Dualshock 4 controller (or your preferred gamepad):

In Windows Settings > Bluetooth & other devices
Add Bluetooth or other device > Bluetooth
On DS4 press SHARE and PS/home button at the same time and hold until controller LED starts flashing
Should appear as “Wireless Controller”, click it to pair

To only see games with full gamepad support:

Steam Library
Move to the far right to find Filter Games and check Controller Supported

Optional but useful, there’s a number of wireless keyboards with integrated touchpad on the market:

To invert the two-finger scroll gesture, try using registry editor to change ScrollDirection in HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\PrecisionTouchPad. If that doesn’t work, try setting FlipFlopWheel as per this answer.

To enable/disable edge gestures, use EnableEdgy in PrecisionTouchPad registry, or follow this forum post.

Next Steps

Need to find some kind of case. Haven’t decided if I’ll try to buy or 3D print one.

Haven’t been able to get it waking from USB. I’ve tried disabling “Allow the computer to turn off this device to save power” and enabling “Allow this device to wake the computer” for everything related to USB and HID devices in Device Manager, as well as disabling “USB selective suspend” in advanced Power Settings. In the BIOS the only options for Advanced > ACPI Settings > ACPI Sleep State are “Suspend Disabled” and “S3”, but there doesn’t seem to be anything related to USB wake.

Maybe I’ll look into running some other software like RetroArch, or Kodi (formerly XBMC) later.

Most importantly, play some games!

How do YOU moderate dev.to?

jeikabu — Thu, 08 Apr 2021 07:39:31 +0000

Dev.to has "trusted users" that can moderate posts/comments.

If you've got this, how have you used it, or when do you not use it?

Early Raspberry Pi Pico and Rust

jeikabu — Sun, 04 Apr 2021 00:00:00 +0000

The Raspberry Pi Pico announced earlier this year is a $4 microcontroller board from the makers of the ever-popular Raspberry Pi. As a Rust-enthusiast one of first questions I tend to investigate is:

Can it run Rust?

TL;DR Yes, but it’s not quite ready for prime-time. Yet.

Hardware

The Pico utilizes the (also new) RP2040 microcontroller. Compared to the already minimalist Pi Zero, it has even more modest specs:

Dual-core Arm Cortex M0+ processor at up to 133 MHz
264KB of SRAM, and 2MB of on-board Flash memory
USB 1.1 with device and host support
26 × multi-function GPIO pins
2 × SPI, 2 × I2C, 2 × UART, 3 × 12-bit ADC, 16 × controllable PWM channels
Temperature sensor
Accelerated floating-point libraries on-chip
8 × Programmable I/O (PIO) state machines for custom peripheral support

Rust Demo

First, download and install ARM toolchain for the host system. If necessary, add it to PATH environment variable. For example, if you install the .pkg on a Mac you’ll need to add /Applications/ARM/bin/.

Next, setup rest of build environment and build a simple demo:

# Add support for ARM Cortex M0+
rustup target add thumbv6m-none-eabi

# Build pico-sdk for elf2uf2 binary
git clone https://github.com/raspberrypi/pico-sdk --depth 1 --recursive --single-branch
pushd pico-sdk && mkdir build && cd build
cmake .. && make
# Output: build/elf2uf2/elf2uf2 (executable)
popd

# Build sample program and convert output from elf to uf2
git clone https://github.com/rp-rs/pico-blink-rs
cd pico-blink-rs
cargo build --release
../pico-sdk/build/elf2uf2/elf2uf2 target/thumbv6m-none-eabi/release/pico-blink-rs pico-blink-rs.uf2

rp2040_pac seems to have changed since the example was written, so cargo build fails with “error[E0609]: no field gpio25 on type PADS_BANK0”. In main.rs change:

     // Configure pin 25 for GPIO
- p.PADS_BANK0.gpio25.write(|w| {
+ p.PADS_BANK0.gpio[25].write(|w| {
         // Output Disable off
         w.od().clear_bit();
         // Input Enable on
         w.ie().set_bit();
         w
     });
- p.IO_BANK0.gpio25_ctrl.write(|w| {
+ p.IO_BANK0.gpio[25].gpio_ctrl.write(|w| {
         // Map pin 25 to SIO
- w.funcsel().sio_25();
+ w.funcsel().sio_0();
         w
     });

Finally, flash the uf2 (like with MicroPython or C/C++):

Push and hold the BOOTSEL button on the Pico and plug into host USB port
- Pico should mount as Mass Storage Device “RPI-RPI2” on host
Copy UF2 file to “RPI-RP2” volume
Pico will reboot and run program

Coming Soon

main.rs is pretty… low-level. To address this, rp-hal is in progress to implement the Rust Embedded HAL. Once that is complete, the same example might be written like a similar example for the Adafruit METRO M0:

#[entry]
fn main() -> ! {
    let mut peripherals = Peripherals::take().unwrap();
    let core = CorePeripherals::take().unwrap();
    let mut clocks = GenericClockController::with_external_32kosc(
        peripherals.GCLK,
        &mut peripherals.PM,
        &mut peripherals.SYSCTRL,
        &mut peripherals.NVMCTRL,
    );
    let mut pins = hal::Pins::new(peripherals.PORT);
    let mut red_led = pins.d13.into_open_drain_output(&mut pins.port);
    let mut delay = Delay::new(core.SYST, &mut clocks);
    loop {
        delay.delay_ms(200u8);
        red_led.set_high().unwrap();
        delay.delay_ms(200u8);
        red_led.set_low().unwrap();
    }
}

Nowhere close to as svelte as the MicroPython version, but then Rust isn’t particularly well known for being terse.

Heterogeneous Smart Home Lighting (Part 1)

jeikabu — Sun, 07 Mar 2021 00:00:00 +0000

I’ve been messing around with my “smart home” setup for a while now. I started with Philips Hue for no other reason than it seemed to be the leader or de-facto standard for smart lighting. But, let’s be honest- Hue isn’t cheap, even for the white (non-colored) bulbs; they’re 12-18 EUR or 12+ USD a piece depending on the bundle.

The other day I came across this post about using Ikea brand “Trådfri” smart lights (which start at 10 EUR/USD) alongside Hue. On a whim I picked up a few different models and following the guide:

Hue App > ⋯ > Light Setup > Add light
Power-cycle the light 6 times

They all showed up like a charm and worked like the rest of my Hue lights. Even the LED Power Supply. I was able to add them to existing rooms, and they even went straight into Home Assistant.

However, complications arose when I tried to use non-Zigbee devices (e.g. bluetooth/Wifi devices by Wiz, etc.), or non-lights like Ikea power control or Xiaomi motion sensor.

The Ikea Trådfri motion sensor is a bit odd because while you can pair it with 1 or more lights that it simply turns on, that hardly qualifies as “smart”. I’d like to use them like more generalized sensors. I’m particularly excited by the Wiz motion sensor because while larger than some of the other options it can use standard AA rechargeable batteries instead of obnoxious batteries like CR2032:

Turns out a lot of these failings can be solved with software (hurray!), but I leave that for my next post.

DEV Community: jeikabu

Cross-posting to Tumblr with AWS Lambda

Tumblr API

OAuth

Client

Callback

Basic Lambda

API Gateway

Final Lambda

Posting to Tumblr

Yet Another Snazzy Rust CLI

Devto and REST

Medium and REST

Hashnode and GraphQL

Miscellany

Async

Error Handling

Logging

Please stop the forem app pop-up

Benchmarking in .NET or How I Learned to Stop Worrying and Love Perf Regression Testing

Basics

More

Reports

Comparisons

CI

Physical Security

Locks and Lock Picking

Does Physical Security Really Exist?

Native Code in .NET 5.0 and C# 9.0

NativeLibrary

Function Pointers

Comparison

3D Printing a Custom Device Case

Creating 3D Models

Modeling/FreeCAD

Slicing and Printing

Optional (But Often Useful)

Final Product

Rust in Android OS and Linux

Building an AMD Ryzen APU (mini) Game Machine

Hardware

Benchmarks

Software

Next Steps

How do YOU moderate dev.to?

Early Raspberry Pi Pico and Rust

Hardware

Rust Demo

Coming Soon

Heterogeneous Smart Home Lighting (Part 1)