DEV Community: Brodan

Learning NFT Provenance by Example: A Bored Ape Investigation

Brodan — Wed, 23 Feb 2022 03:57:46 +0000

NOTE: This post was originally published on my personal blog and can be found here.

I was recently approached by an artist friend of mine to help them build and launch an NFT collection using their artwork. I've never worked with smart contracts or any of the NFT-related technologies, so I've been learning all of it myself. One concept that I found particularly interesting was the idea of provenance.

This post will cover provenance as it pertains to NFTs. It's broken up into two sections. The first section discuses my journey of learning about and calculating the provenance of a popular NFT collection. The second section discuses the curious discrepancy I found in my calculation, how I found it, and what it could mean.

What is Provenance?

The term "provenance" has existed long before smart contracts and NFTs. In general terms, it refers to the place of origin or earliest known history of something. The term is almost synonymous with "origin". In the crypto/NFT space, provenance could actually mean a few things, but it's usually referring more specifically to a provenance hash, which is a hash of all the assets of an NFT collection before minting.

The purpose of a provenance hash is to show that the assets that exist on-chain are the same assets that were initially generated. If any of the assets are manipulated in any way, it would be obvious because re-calculating the hash would result in a value different than the provenance hash.

One of the current issues with a provenance hash is that there is no standardized/consistent method of calculating it, so one collection might implement it differently than the next. It's up to the NFT creators to describe how their particular provenance hash was calculated so that it can be independently verified.

Armed with some conceptual knowledge of provenance, I decided to solidify my understanding of it by verifying the provenance of an existing NFT collection. I chose the Bored Ape Yacht Club (BAYC) collection to test on because of its ubiquity and because their provenance information is available publicly on their site (many NFT collections do not provide this information).

How is Provenance Calculated?

The code used in this section can be found in my bayc-provenance-verifier repo.

The BAYC provenance page describes exactly how their provenance hash was calculated:

Each Bored Ape image is firstly hashed using SHA-256 algorithm. A combined string is obtained by concatenating SHA-256 of each Bored Ape image in the specific order as listed below. The final proof is obtained by SHA-256 hashing this combined string. This is the final provenance record stored on the smart contract.

They also provide their final proof hash of cc354b3fcacee8844dcc9861004da081f71df9567775b3f3a43412752752c0bf.

The instructions above seemed simple enough to reproduce so I decided to do so using TypeScript. I used the following code:

import * as crypto from 'crypto'
import axios from 'axios'

type BAYCData = {
    provenance: string;
    collection: Metadata[];
}

type Metadata = {
    tokenId: number;
    image: string;
    imageHash: string;
    traits: object;
}

const BAYC_PROVENANCE = 'cc354b3fcacee8844dcc9861004da081f71df9567775b3f3a43412752752c0bf'
const BAYC_METADATA_URL = 'https://ipfs.io/ipfs/Qme57kZ2VuVzcj5sC3tVHFgyyEgBTmAnyTK45YVNxKf6hi'

async function main() {
    const fetchImage = (url: string): Promise<string> => new Promise(async (resolve) => {
        const image = await axios.get(url, { responseType: 'arraybuffer' })
        const hashedImage = crypto.createHash('sha256').update(image.data).digest('hex')
        resolve(hashedImage)
    })

    const response = await axios.get(BAYC_METADATA_URL)
    const data = await response.data as BAYCData
    const hashes: string[] = []

    // get every individual image hash
    for (let index = 0; index < data.collection.length; index++) {
        const hash = await fetchImage(data.collection[index].image)
        hashes.push(hash)
    }

    // make one big string and hash it
    const concatenatedHashString = hashes.join('')
    const result = crypto.createHash('sha256').update(concatenatedHashString).digest('hex')

    // check for a match
    console.log(result === BAYC_PROVENANCE ? 'SUCCESS' : 'FAILURE')
}

main()
    .then(() => process.exit(0))
    .catch((error) => {
        console.error(error)
        process.exit(1)
    })

This script can be ran using ts-node.

As a warning: this script takes a long time to run. I didn't realize just how large of a number 10,000 is, but it takes a while to make all of those requests since this is being done synchronously and in serial.

My first attempt at writing this script was less synchronous and used Promise.all() to make the individual image requests concurrently, but it would always fail around the 200th request due to IPFS gateway issues. I suspect it was because of the huge amount of simultaneous outgoing requests being made.

My Results

I ran the script, waited a while, and saw the FAILURE message log to my console. Then, I double checked my logic and ran it again. FAILURE. I repeated this a few times. I had no idea why it wasn't working. The logic looked fine. I decided to compare the two concatenated hash strings instead.

Finding the Incorrect Hashes

I first tried to diff the concatenated hash string that I produced with the one listed on the site's provenance page, but since the output is only one line it doesn't tell me anything other than that the lines are, obviously, different.

After some googling I came across the fold command which can "fold long lines for finite width output device". SHA-256 hashes are 64 hex characters long, so the -w 64 flag will ensure that every individual hash from the concatenated hash is split onto its own line for the diff.

Now I could use the diff command and the fold command together to figure out where the two hash strings differed:

diff <(fold -w 64 my_hash.txt) <(fold -w 64 bayc_hash.txt)

The command above uses process substitution to return the output of fold as a file-descriptor so that it can be used as the arguments for diff. Here is a snippet of what the output looks like:

2032c2032
< e33123649788fb044e6d832e66231b26867af618ea80221e57166f388c2efb2f
---
> 5bb1c8a8390c284e9a4634c04eee34dfd08759d66d2b613b0631ab10e2f1f3d9
2034c2034
< 00ebcc4a1409783b3809a0ff383d38a54bc768aa6ff5490e008e788660a85fbd
---
> 5bb1c8a8390c284e9a4634c04eee34dfd08759d66d2b613b0631ab10e2f1f3d9
2039c2039
< d1ec635d95f689eb22191daba2b03efb092d1d8e56e9a09ad697f45c0dd1d014
---
> 5bb1c8a8390c284e9a4634c04eee34dfd08759d66d2b613b0631ab10e2f1f3d9
2041,2042c2041,2042
< 6c62372729385006372dd94807cbcc3d6bd3e4e6d84f349790b031de2bc318f4
< f0455b3cb1998c00ab6ea3bbdd74939aeaea08ab11bc79567d045e612882c6d6
---
> 5bb1c8a8390c284e9a4634c04eee34dfd08759d66d2b613b0631ab10e2f1f3d9
> 5bb1c8a8390c284e9a4634c04eee34dfd08759d66d2b613b0631ab10e2f1f3d9

The output is grouped together so that the line number(s) that differ are shown first, e.g. 2032c2032 and then difference between those lines.

Something Suspicious

Notice anything strange about the output above? It took me a second but then it hit me.

Nearly every line in the diff (31 out of 32) shows the same hash: 5bb1c8a8390c284e9a4634c04eee34dfd08759d66d2b613b0631ab10e2f1f3d9

Don't believe me? Go to the BAYC provenance page and cmd + f the hash above. It exists 31 times in their provenance.

How could that be possible? It's easy to miss something like this when glancing at the provenance page because there are 10,000 lines of hashes listed, but this is clear as day.

A Google search of this hash returns only one result, which is the provenance page for another NFT project called BearNBear. And just like the BAYC, the image shown in that collection does not actually produce this hash when run through SHA-256. How on Earth is it possible that two completely unrelated NFT projects both mention this hash?

A friend of mine, Matt, stepped in to help figure out the significance of this hash but we remain stumped. Another colleague of mine, @conundrumer, suggested that this hash may be that of a null value, but we quickly determined that this was not the case either.

Another Outlier

Finally, there was one hash out of the 32 mismatches that wasn't the same value as the rest of them:

2306c2306
< d68d5402848bb7fba993411e521e1352c3446cd6e6676c95fa55d3e51061d540
---
> d1d7cec7d41312780f1e5064fd2d199a7bef312ec84c8ceb367005bbb26f5d7a

This, to me, is even more fishy than the others, and could mean that Bored Ape #1159 was manually manipulated at some point after minting.

What Does it All Mean?

Where did these incorrect hashes come from? What image produces the hash that is listed 31 times? How has no one else noticed this before me?

Frankly, I don't know the answer to any of these questions. It's possible that there were some issues that came up during BAYC's image generation but were resolved before minting began. However, it is also possible that this means that assets were manipulated at some point after being minted, which would be extremely sus.

The former seems much more likely. Nevertheless, it's pretty suspicious and I really wish I could get an explanation directly from someone at Yuga Labs. It feels so wrong that I'm the first person to raise a conern about this.

The Bored Apes in question, based on my calculated hashes and accounting for the starting index offset of 8853, are numbers 885, 887, 892, 894, 895, 897, 1087, 1089, 1091, 1094, 1109, 1111, 1112, 1113, 1118, 1120, 1121, 1126, 1129, 1134, 1136, 1140, 1142, 1153, 1155, 1159, 1199, 1200, 1303, 2183, 5250, 8817.

Wrapping Up

I had a lot of fun learning about provenance and provenance hashes in this manner. Hopefully you found something of value in this post. If you'd like to follow along with the drama, I recently tweeted at the BAYC account in the hopes that someone in the community would see it and provide an explanation (or at least point me to someone who can). Any likes/retweets/follows are appreciated!

// Detect dark theme var iframe = document.getElementById('tweet-1491091395645689860-372'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1491091395645689860&theme=dark" }

Thanks for reading!

", hopefully" - An Intro to Git Hooks

Brodan — Thu, 06 May 2021 19:43:27 +0000

In yet another instance of a blog post inspired by a single tweet, this post will offer a very basic introduction to git hooks (commit hooks, to be specific).

The Tweet

// Detect dark theme var iframe = document.getElementById('tweet-1389708730086957061-448'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1389708730086957061&theme=dark" }

I saw this on my timeline earlier today, had a good chuckle, and then realized that despite knowing about git hooks for a long time, I've never actually written one. I decided to fix that by writing a dead simple hook that would recreate the aforementioned Tweet.

The Code

Start by initializing a new git repo as such:

cd /tmp && mkdir hook-testing
cd hook-testing
git init

git actually generates a number of sample hooks when you initialize a repo. You can see them all by running ls .git/hooks/.

If you look closely, there's a hook named prepare-commit-msg.sample. How convenient! This hook, once enabled, will run every time a git commit command is run when working in this repo.

You can read more about this hook in the githooks Documentation.

In order for git to actually pick up and run a hook, the .sample extension must be removed:

mv .git/hooks/prepare-commit-msg.sample .git/hooks/prepare-commit-msg

Open .git/hooks/prepare-commit-msg in an editor and feel free to look at the examples. Then replace it all with the following:

#!/bin/sh
COMMIT_MSG_FILE=$1
COMMIT_SOURCE=$2

DREAM=", hopefully"
if [[ "$COMMIT_SOURCE" == "message" ]]
then
    echo `cat $COMMIT_MSG_FILE`$DREAM > $COMMIT_MSG_FILE
fi

I kept this hook pretty simple since my shell-scripting abilities are lackluster.

git passes three arguments into the prepare-commit-msg hook, but we only care about the first two:

$1 is the name of the file that contains the commit log message. We will append our optimistic message to this file.
$2 is the source of the commit message and is set according to how the commit is being generated (such as in a merge, squash, etc, or just a regular old commit).

In this case, the hook is only going to run if the commit source is "message", meaning that the commit was made using the -m flag. Feel free to modify this to your liking.

In order to see it in action, we need to commit something:

git commit --allow-empty -m "adding an empty commit"
[master 1031a40] adding an empty commit, hopefully

As you can see above, the commit message was updated to include the ", hopefully" message. You can run git log to see it again if you want to double-check.

The Conclusion

I hope you found this post informative and entertaining. The hook itself is very simple but I actually learned a log about git internals while working on it.

If you'd like to see the other posts I've written that were inspired entirely by Tweets, consider these:

Thanks for reading!

"Grab these vehicles": Send Real Simeon Texts Using Twilio Functions

Brodan — Thu, 11 Jun 2020 04:16:45 +0000

I started playing Grand Theft Auto Online again after putting it down for several years and I was quickly reminded of one of the most annoying features of the game: Simeon vehicles.

Also known as Simeon Car Export Requests, fans of the game are all too familiar with this mechanic, which has achieved meme-status due to its annoyance and inconvenience.

I was inspired to build this functionality IRL to troll my GTA Online friends with real text messages. This post will quickly walk through how to build your own Simeon texting service in less than 10 minutes using Twilio Functions.

Getting Started

If you've never used Twilio before and you're interested in getting started, consider using my referral code for $10 of free Twilio credit (which goes a really long way and will cover anything needed for this post).

Twilio Functions allow you to quickly and easily execute JavaScript in a serverless environment in response to incoming messages or voice calls (or HTTP requests). Functions are super extensible and can be integrated into other Twilio offerings as well, but I find them best used for quick and silly projects like this one that don't warrant an entire back-end and deployment pipeline.

The first step is to buy a Twilio phone number, which is needed to send outgoing SMS messages. This can be done from the Twilio Console. More info on buying a Twilio number can be found here.

Configuring the Twilio Function

Next, navigate to the Functions section of the Console and create a new function. Give the function a descriptive name and path (I've obscured mine for privacy's sake). For simplicity' sake, make sure that 'Check for valid Twilio signature' is off, since requests to this Function will come from HTTP and not other Twilio services. More information about this can be found in Twilio's Security documentation. The configuration should look similar to this:

The next step is to paste in the following code into the Function:

exports.handler = function(context, event, callback) {
    messageOptions = [
        'Cheval Surge, Ocelot Jackal , Obey Tailgater, Dundreary Landtstalker, Maibatsu Penumbra.',
        'Dundreary Landstalker, Maibatsu Penumbra, Ocelot F620, Fathom FQ 2, Mammoth Patriot.',
        'Dundreary Landstalker, Maibatsu Penumbra, Ocelot F620, Fathom FQ 2, Bollokan Prairie.',
        'Karin BeeJay XL, Bravado Gresley, Albany Buccaneer, Western Daemon, Western Bagger.',
        'Ubermacht Sentinel XS, Vapid Dominator, Benefactor Schafter, Cheval Surge, Ocelot Jackal.',
        'Benefactor Serrano, Mammoth Patriot, Emperor Habanero, Schyster Fusilade, Bravado Gresley.',
        'Fathom FQ 2, Mammoth Patriot, Emperor Habanero, Schyster Fusilade, Bravado Gresley.'
    ]
    randomOption = messageOptions[Math.floor(Math.random() * messageOptions.length)]
    messageBody = "Grab these vehicles: " + randomOption
    context.getTwilioClient().messages.create({
        to: event.toNumber,
        from: context.SIMEON_NUMBER,
        body: messageBody
    }).then(msg => {
        callback(null, msg.sid);
    }).catch(err => callback(err));
};

Remember to save the Function after pasting.

The code above will choose a random text message body based on the possible texts that Simeon sends in-game and then send an outgoing SMS via the built-in twilio-node helper library.

Two of the more notable lines are event.toNumber and context.SIMEON_NUMBER.

event.toNumber is looking for a query param called toNumber which must be included on incoming requests (an example is shown in the next section).
context.SIMEON_NUMBER is looking for an environment variable named SIMEON_NUMBER configured within the Function console.

Set the Environment Variable

The final step is to set this SIMEON_NUMBER environment variable for the so that the Function can access your Twilio phone number it without having to hard-code it. Navigate to the Function Configuration page and add a new environment variable named SIMEON_NUMBER and set the value to the number that was purchased earlier (make sure to enter it in E.164 format).

Testing the Twilio Function

The way to trigger this Function is via an HTTP request. cURL can be used to trigger the Function from the command line as seen here:

curl -X POST https://xxxxxx-yyyy-0000.twil.io/simeon?toNumber=15555555555

Replace the URL with the one from your own Function and set the toNumber param to the phone number you want to message.

Wrapping Up

That's all there is to it. The Function can be triggered from any manner such as the command line, a Node back-end, scheduled on a cronjob, etc. The possibilities with Twilio Functions and how to trigger them are endless.

I hope you enjoyed following along with this silly attempt at video game humor. If you have similar ideas for Functions based projects post them in the comments section!

Thanks for reading!

Waifu MMS Bot - Send a Selfie, Receive a Waifu

Brodan — Fri, 01 May 2020 06:24:40 +0000

Welp, I started this about 6 hours ago after the hackathon deadline completely sneaked up on me. In true hackathon fashion: the code ain't pretty, it was submitted an hour before the deadline, and by some miracle it works!

This project was inspired by a tweet I saw this afternoon:

// Detect dark theme var iframe = document.getElementById('tweet-1250427941567094786-103'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1250427941567094786&theme=dark" }

As well as my friend and Twilio Evangelist Sam Agnew's response:

// Detect dark theme var iframe = document.getElementById('tweet-1255867485640888320-714'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1255867485640888320&theme=dark" }

What I built

This application allows you to send a selfie to a Twilio phone number and receive and "waifu" version of it in return that was generated using a trained model. The image conversion is all handled by the Selfie 2 Waifu project which was built by creke. I simply integrated Twilio and built the automation around it.

Category Submission:

I could only really see this project falling under either the 'Interesting Integrations' or 'Exciting X-Factors' categories.

Demo Link

Since I waited until (literally) the last minute to build and submit this I didn't have enough time to record a proper video demo or anything but here's a sample interaction with this app from my phone:

I also encourage anyone to fork the repo and try it out themselves!

Link to Code

The code is MIT licensed and fully available on GitHub along with some (rudimentary) setup instructions.

Brodan / waifu-mms-bot

Generate your waifu-self using Twilio MMS

waifu-mms-bot

Generate your waifu using Twilio MMS. Simply send a selfie to your Twilio number via MMS and receive your waifu in return.

This project was built on April 30, 2020 for the Twilio Hackathon on DEV.

This is a project was built using Express, Twilio, Puppeteer, and most importantly Selfie 2 Waifu.

Development

To run this locally app locally you'll need to do the following (replacing the values as you go):

git clone https://github.com/Brodan/waifu-mms-bot.git
npm install
export TWILIO_ACCOUNT_SID='XXXXXXXXXXXXXXXXXXXXXXX'
export TWILIO_AUTH_TOKEN='YYYYYYYYYYYYYYYYYY'
export TWILIO_NUMBER='+15555555555'

# run this in a in a seperate terminal
# install instructions: https://ngrok.com/download
ngrok http  5000

# copy your ngrok URL and export it
# also make sure configure your Twilio number to point to *YOUR_NGROK_URL*
export NGROK_URL=*YOUR_NGROK_URL*

npm start

Once everything is configured and running, send a selfie via MMS to the Twilio number you configured and wait a few seconds…

View on GitHub

How I built it (what's the stack? did I run into issues or discover something new along the way?)

I built this app using the following tools/technologies:

Selfie 2 Waifu for handling the actual image processing/conversion
Twilio MMS for sending and receiving images
Express for the back-end
ngrok for my localhost tunneling so I could get up and running quickly and not worry about deploying
Puppeteer for headless browser automation

The biggest issue I ran into while building this app was figuring out how to automate the uploading of selfies to the Selfie 2 Waifu web app. I originally started out using an awesome tool called taiko that I usually use for these kinds of interactions. However, due to the page's file upload proccess I couldn't quite get automated uploads working. I then switched to Puppeteer which I had never used before and luckily it was just simple to work with and was able to accomplish what I needed.

The only other issue I ran into was working with asynchronous code throughout my Express server since I am a Node newbie and struggled with some of the async/await syntax. This, along with the time constraints, is why the code is definitely not as clean as it could be and might mortify some of the JS experts.

Additional Resources/Info

In reality, all the credit should for this app should go to the Selfie 2 Waifu author, creke. Without their awesome app and the inspiration it caused I wouldn't have built this.

Additional thanks to DEV and Twilio for running this hackathon.

Configuring a .pypirc File for Easier Python Packaging

Brodan — Sun, 20 Jan 2019 02:25:07 +0000

PyPI is the Python Package Index. Its purpose is to help Python developers find and install software developed by the Python community.

I recently built my first Python package, patter, and released it publicly via PyPI. I ran into a few hiccups along the way, so I am writing this post to help those in a similar position.

This post will describe the basics of a .pypirc file and how to configure and secure it. I originally wrote this post for the Truveris Engineering blog and I am reposting it here for additional reach.

Getting Started

Before proceeding, it's a good idea to make sure that the setuptools and wheel libraries are up to date. The following command will update them if needed:

$ pip install -U setuptools wheel

This post will assume that you have a new Python library that is ready to be published. The source code should be packaged using a command like the one below. Your command may differ slightly depending on the needs of your package.

$ python setup.py sdist bdist_wheel

To read more about creating a distributable Python package, see these docs.

In the next section, I use the twine utility to facilitate the release of my new package. You can read about the benefits of using twine over the built-in packaging tools here. Install twine using the following command:

$ pip install twine

The .pypirc File

There are two main benefits to using a .pypirc file:

It removes the need to enter a username/password when pushing to PyPI.
It simplifies command line usage when pushing packages to a non-default package repository (i.e. anywhere other than pypi.org).

The official documentation on the .pypirc file can be found here. The contents of my .pypirc file can be seen below. This file must be placed in $HOME/.pypirc for pip/twine to use it.

    [distutils]
    index-servers=
        pypi
        testpypi

    [pypi]
    username: brodan
    password: xxxxxxxxxxxxxxxx

    [testpypi]
    repository: https://test.pypi.org/legacy/
    username: brodan
    password: yyyyyyyyyyyyyyyy

Keep in mind, pypi.org and test.pypi.org are not integrated, so you'll need to have a separate account created on each site.

One thing to notice above is that the [pypi] section does not have repository configured, but the testpypi section does. That is because the repository variable defaults to https://upload.pypi.org/legacy/, so it does not need to be included in that section.

Uploading Python Packages

Once the file above is in place, the --repository flag can now be used with twine to specify which package repository your packages will be uploaded to:

If you wish to upload a package to the TestPyPI repository, the following command should be used:

$ twine upload --repository testpypi dist/*

Similarly, once the package is ready to be released to the public, the following should be used:

$ twine upload --repository pypi dist/*

Notice that you won't be prompted for a password when running either of the above commands. You also no longer need to copy and paste repository URLs into the terminal.

Securing The .pypirc File

Since the .pypirc file is storing sensitive information (i.e. passwords) in plain text, it's important to set the permissions on this file accordingly so that other users on the system can't access this file.

To do this, run the following command:

$ chmod 600 ~/.pypirc

The command above will ensure that only the file owner (which should be your own user) can read and write to this file. Additional info on file permissions in UNIX can be found here. Thanks to this StackOverflow answer for help on this section.

Wrapping Up

With a .pypirc file in place the process of pushing Python packages to public repositories is now much easier.

If you have any questions or feedback regarding this post, please reach out to me via email or Twitter. Thanks for reading!

How Do I Interview Candidates as a Junior Engineer?

Brodan — Thu, 09 Nov 2017 19:21:04 +0000

I'm a junior engineer and I've been at my first full-time gig for ~7 months now. My boss/CTO asked me if I'd want to start interviewing new engineering candidates and I expressed interest, but now I'm starting to worry that I'm not qualified to interview anyone since I'm just starting my career.

What should be expected of me as an interviewer and how can I prepare myself to interview candidates who are anywhere from junior-senior level?