DEV Community: Javier Gonzalez

ECDSA in Bitcoin

Javier Gonzalez — Tue, 19 Apr 2022 18:22:17 +0000

Here is this article in short video form:
https://www.youtube.com/watch?v=tl6RGKKvHto

Since I never took computer science courses, I have to catch up with some of the fundamentals, and I find them fascinating. One such fundamental of computer science is cryptography, and today I want to go through how it is applied to generate wallets in Bitcoin from scratch.

The math used to generate a wallet from scratch is called Elliptic Curve Cryptography. It used in Bitcoin and is crucial to maintaining the integrity of the protocol. The amazing part is that it is so secure, that you don't have to even be connected to the internet because this process is just based on math.

Let’s get into how it works.

Everything is based on this curve called the Secp256k1 derived by the function

y = sqrt(x**3 + 7)

To use it, we generate any random number up to 2^256 and multiply it by a constant point G in the curve

The point G is constant for every bitcoin address and here it is as x and y coordinates:

The math that is more interesting and we are more concerned about is how to generate a public key from ANY random number. And that math is a simple function here:

K = k  * G

where k is the private key, G is a constant point called the generator point, and K is the resulting public key.

k can be anything, the important part is that it is random. Once you have a random value for k you just multiply that by G which is the process of extending a tangent until it intersects with a different point in the line and reflecting that point across the X axis.

The result is called your public key and can be shared with ANYONE because it is extremely improbable that someone can derive the private key backwards from it.

You still have to run it through some hashing functions to generate an address, but this is pretty much it. This math has been around since the 80s and is considered secure enough to hide government secrets. By signing a transaction with that private key, you are basically proving that you are the only person with access to the private key used to generate that public key.

Solidity Gas Optimizations pt.4 - Efficient Strings

Javier Gonzalez — Sun, 17 Apr 2022 16:35:09 +0000

In the previous lesson, we saw a bit about how the solidty compiler and the EVM stores data in 32 byte 'buckets' or 'slots'. This applies to every operation that deals with storage. This can come in handy when dealing with strings, specially when using revert() or require().

What is a String?

In the Solidity Documentation we can see that:

Variables of type bytes and string are special arrays. The bytes type is similar to bytes1[], but it is packed tightly in calldata and memory. string is equal to bytes but does not allow length or index access.

Basically, strings are arrays of UTF-8 characters, and arrays are basically a fixed length sequence of storage slots located next to each other. Therefore, like everything else we need to think in terms of storage buckets

How to use this to our advantage

This means that when we compile our contracts, we would prefer to use a single slot for the strings we use.

Each character in a string is a UTF-8 Encoded byte, meaning that your strings can be up to 32 characters in length if you do not use specially encoded characters like emojis to be contained in a single storage slot.

A Note About UTF-8
UTF encoding is a standard for how computers represent each character in a language. All the standard English characters take up one byte, but things get a bit more difficult when we start with characters like 💩 which takes up 4 bytes (64 bits).
Here's a great resource to look up how many bytes a string takes up under UTF-8 Encoding. https://mothereff.in/byte-counter

So basically, when we can use this everywhere for example in revert statements, we can think of how to make these messages fit in multiples of 32 bytes.

So instead of having this:

// 40 bytes or two slots
revert(condition, "User has insufficient funds for transfer")

Consider having this:

//32 bytes or one slot
revert(condition, "Insufficient funds for transfer.")

Again, sometimes being verbose is better for everyone to know what is happening, so consider the tradeoffs. This works best when an extra character can add a considerable expense of a function call.

Some further reading on UTF-8 characters: https://blog.hubspot.com/website/what-is-utf-8

Solidity Gas Optimizations pt. 3 - Packing Structs

Javier Gonzalez — Sun, 03 Oct 2021 16:35:26 +0000

So now that we've looked at some of the basics of assembly reads and writes memory, we can begin to look at how that memory is actually stored on each node. Everytime we make a variable in our solidity code, the evm stores it in a storage slot of 32 bytes (256 bits). That means that every time we have a uint (which is read as a uint256) we have packed a storage slot fully.

So lets look at some code:

    uint128 a;
    uint256 b;
    uint128 c;

What the evm does is try to fit everything into storage slots sequentially, but since variable b takes up an entire slot, it cannot fit in in the first one and needs to allocate a total of 3 32byte slots. If you instead order them so that the two smaller slots are next to each other, you can save one such storage operation. A more efficient code would look like this:

    uint128 a;
    uint128 c;
    uint256 b;

That would allow for the EVM to only need to allocate two storage slots and 'pack' your variables.

Variable types

Now let's look at the size of some common data types in Solidity:

uint256 is 32 bytes
uint128 is 16 bytes
uint64 is 8 bytes
address (and address payable) is 20 bytes
bool is 1 byte
string is usually one byte per character

You can further break down a uint into different sizes, uint8, uint16, uint32... just keep in mind that your integer will overflow if you are using solidity version < 0.8.0 or your function will fail if you are using version > 0.8.0.

The largest number is calculated as 2^(number of bits) - 1, meaning uint8 goes up to ((2^8) - 1) = 255 before your function fails.

I suggest keeping these in mind when you are designing structs.

Disclaimers

A few small caveats, this only works when you are using Structs (custom data objects) and/or declaring variables in your contract storage. Function arguments use memory which does not work in this way.

Also, if you will not be packing your structs this way, it actually can be more expensive to use a datatype which is smaller than the 32byte limit! So if you have something like

    uint8 a;
    uint256 b;

It will be cheaper to make both variables uint256! Counter intuitive, I know!

The reason for this is that the EVM reads 32 bytes at a time and will have to do some operations to make the data it is reading go down to 8 bits (1 byte) which is counter productive. Lastly, this only really applies when you are doing multiple read and writes to the same struct or storage slot in the same operation.

When you are writing code, you don't always want to optimize for efficiency, but for developer productivity and readability. Sometimes, it might be OK to do slightly less efficient code if it makes the project easier to maintain.

We will leave this here, stay tuned for part 4!

Solidity Gas Optimizations pt.2 - Constants

Javier Gonzalez — Wed, 29 Sep 2021 06:21:22 +0000

Variables in the blockchain

Storing variables in memory, on a normal computer, is easy and cheap. However, blockchains are distributed systems, in which every node - or computer - has to store the data locally. That makes storing data expensive, this behavior is discouraged except where necessary.

SSTORE and SLOAD

These are two of the more expensive OPCOs in the EVM for the reason outlined above. In my previous post I talk a bit about the gas cost of loading from storage using SLOAD, and why it is better to use MLOAD instead when possible.

However, there are times that you need to instantiate a variable on creation or deployment and do not expect that variable to change. In these cases, you can use a constant or immutable modifier which will let the solidity compiler know about the future of that variable. Let's take an example simplified contract:

contract Token {
    uint8 VERSION = 1;
    uint256 decimals;
    constructor(uint256 val) {
      decimals = val;
    }
}

The decimals variable in the contract is only there for display purposes in the frontend, ERC20 tokens don't actually have a concept of 'decimals'. This means that it does not use it, it is only for users of the contract to know how to format outputs. This also means that the variable should not change. To indicate that it should not change, we have two options, constant or immutable. According to the docs:

The compiler does not reserve a storage slot for these variables, and every occurrence is replaced by the respective value.

Constant variables are replaced at compile time by their values, while immutable variables are replaced at deployment time. Either way, we avoid the annoying fees required with doing an SLOAD operation.

One simple fix looks like :

contract Token {
    uint8 constant VERSION = 1;
    uint256 immutable decimals;
    constructor(uint256 val) {
      decimals = val;
    }
}

One small caveat to note is that constant variables cannot make reference to the state of the blockchain. You cannot do something like:

uint256 constant VERSION = block.number

or things to do with context generally, like block.timestamp, address(this).balance or block.number msg.value or gasleft() nor call external contracts.

Immutable variables however, are fine to do so.

We'll leave it here, and stay tuned for part 3!

Solidity Gas Optimizations pt.1 - Memory vs Storage

Javier Gonzalez — Thu, 09 Sep 2021 10:05:32 +0000

Building on a blockchain

If you are familiar with a language like JavaScript, you tend to never think about how your variable is stored, except to deal with the scope of the variable. When you are making programs to run on a distributed system like a blockchain, you have to think about things a bit differently.

Solidity works as a compiled language where each operation gets converted to a lower level opco which the EVM can understand and interpret. Then, every operation that you write on your program gets executed on every computer in the network, which is why every operation costs 'gas' to prevent spamming and more importantly, infinite loops. In solidity, getting to know the machine readable operations and their associated cost literally saves you money.

So let's take an example:

    uint256 percentage = 30;
    function splitAmountToOwnerAndSeller(uint256 amount)
        internal
        view
        returns (uint256 amountForSender, uint256 amountForOwner)
    {
        amountForSender = (amount * (100 - percentage)) / 100;
        amountForOwner = (amount * percentage) / 100;
    }

This functions calculates how much ether should go to the owner and to the seller of an item. The owner gets a percentage defined by the storage variable called percentage. If we break it down into how this works, you actually need to read from the storage variable twice. In some languages that might not be a problem, however if you understand how data is stored on the blockchain, you would realize that reading the variable for amount is an in-memory operation, while reading the variable percentage is a storage operation. It is a different assembly code.

EVM Assembly

Whenever you read the variable percentage you are getting data from the blockchain database (A network of computers that each have to validate that piece of data), and this is done through an opco called SLOAD which according to the Ethereum Yellow Paper costs 800 gas to execute:

Because you do this twice, you would end up spending 1600 gas on reading this variable. To combat that, you could always store the object in memory, and load it from there, which is much cheaper (around 3 gas). So what you could do is write from storage to memory once (SLOAD + MSTORE) = 803 gas, then read the memory variable twice (MLOAD + MLOAD) = 6 gas, for an almost 50% gas reduction for that transaction (1).
So, the code would look like this:

    uint256 percentage = 30;
    function splitAmountToOwnerAndSeller(uint256 amount) internal view returns (uint256 amountForSender, uint256 amountForOwner)
    {
        uint256 ownerPercentage = percentage;
        amountForSender = (amount * (100 - ownerPercentage)) / 100;
        amountForOwner = (amount * ownerPercentage) / 100;
    }

This is especially necessary when you are dealing with loops that read from state regularly, always cast a state variable to memory before entering a loop.

NOTES:

(1) Thanks to user CPlusPlusDeveloper in reddit for pointing out that this is not entirely true, ever since EIP-2929 the first SLOAD operation costs 2100 gas, but once that memory is read, it is cached and considered considered warm, which has a cost of 100 gas to load again. It would still save gas to load and retrieve that variable in memory, especially if read more than twice.

Javascript vs HTML standard form behavior

Javier Gonzalez — Mon, 08 Feb 2021 04:38:23 +0000

Forms are useful. They allow us to pass data to a website. We see them everywhere, signup forms, contact forms, and in a million other ways. They are an html way to communicate with other html pages over HTTP.

You can see how your document tries to communicate with itself with this simple and ugly form.

<form>
  <input name='submission' type='text'>
  <input type='submit'>
</form>

If I type the word 'something' then press submit, I get this:

My URL is updating on form submission! In technical terms, my form is submitting a GET request with the query parameter "submission" set to "something" to the server. This is great for dumb pages or server side rendered applications, but we frontend developers love to handle state on the browser. We could be having state that needs to be persistent, and refreshing the page is unacceptable. (think of a modal popup window with a form, you don't want to refresh the page and make the user re-open it).

So why does the browser refresh happen? Basically, forms in the browser want to pass data to a page. When you submit a form, it redirects you to the page that you submitted the data to. If you don't supply an action attribute to the form, or a method it will submit it to your current page using a GET parameter like we showed above.

so the HTML code above is doing the same as this.

<form action="/" method="GET">
  <input name='submission' type='text'>
  <input type='submit'>
</form>

The form [action="/"] attribute is telling the browser that we want to post the data to the root URL of the website. The method is expecting an HTTP verb (POST, PUT, PATCH, DELETE, etc...) but GET is the default one. GET requests are appended to the url as shown in the image above. This is handy, but often you don't want that because you don't need that information in your browser history for everyone to see.

When you submit a form, the browser navigates to the url that you provide to the [action] attribute. However, we said we wanted to avoid page reload all together. To do this, you need to intercept the standard form behavior with javascript. You can actually do this by using the preventDefault() method on the submit event itself.

<form action="/" method="GET" onsubmit="event.preventDefault()">
   <input name='submission' type='text'>
   <input type='submit'>
</form>

If you run test again, you no longer get a query string in the URL and your state persists!

(This is a boring image but it shows the form element still holding a value because the page didn't refresh!!)

You are preventing the browser from doing what it does whenever it sees a form element.

Now let's handle it with javascript because we all know that you should handle everything with javascript (I'm just leaving this here to piss off backend developers). You can do that without writing it in the html by handling it in your js like this:

 <form>
    <input name='submission' type='text'>
    <input type='submit'>
 </form>

<script>
  document.querySelector("form").addEventListener("submit", (event) => event.preventDefault());
</script>

Ahh much nicer, we took away the native properties of the form element and told the browser, we'll take it from here. Now we can decide exactly how we want the form to behave.

Why use forms at all?

So the more adept programmer might say something along the lines of 'if I have to jump through hoops when I put a form element on the page, can't I just omit it altogether?'. And yes, they would be right that they could do the same as the example above only using an input element and a button. However, forms are the 'semantic' way of writing html, which is useful especially for screen readers. Screen readers are all programmed to react predictably to a form element, but if you don't group the functionality of an input with a submit, you'll have to keep track of all that yourself through adding labels and grouping the behavior together. This is possible, but it is not recommended because it introduces more surface area for mistakes appear.

So there you have it. Hopefully this helps if you are wondering why you are losing the state of the application every time you hit submit on your forms. Stay tuned for more tips and tricks!

I am also self taught

Javier Gonzalez — Tue, 12 Jan 2021 21:55:48 +0000

I taught myself how to code. But I taught myself everything I know. I am not trying to sound full of myself, I actually think that this is true for everyone.

What is learning?

I love the American Pragmatist Philosopher John Dewey. He was much smarter than I. He also thought very deeply about the question of what it is to "learn" something. He developed a theory that says something that might sound basic. To know something is to have an experience of something. He laid the theory that knowing something is being able to apply it in the context in which it will be used. And, the only way to know something is to have done it before, to feel the nuances based on your experiences, to gain the habit, the muscle memory, the traps and the ability to figure out what will probably work and what wont. This also means that knowledge cannot really be taught.

The concept is very similar to what we hear all the time in programming communities. Do projects, build things, don't get stuck in tutorial purgatory, make projects apply what you learned. Dewey would agree with this.

Saying that you can't be taught something sounds harsh. Of course there is value in teaching and learning from others. After all, there will always be people more experienced than you at what you are trying to do. Their experience is absolutely valuable, and what you get from them is knowledge about perils, it is knowledge about what will give you more leverage, about where to direct your attention to get a good basis, but the problem is when you begin to confuse hearing facts about something with learning about something. You cannot transfer knowledge into your head. You need to practice in the context as close as the real thing. Otherwise, you will only get good at answering quizzes, not at solving problems.

So, we are all self taught, because it is only by doing the work that you learn. And doing the work, takes effort that you can simply not buy. A teacher can show you the way, but learning happens on your own.

Do you agree?

Async JavaScript

Javier Gonzalez — Mon, 11 Jan 2021 00:21:16 +0000

JavaScript is a single threaded language.

That means that JS only does one thing at a time. This is key to understanding how it works.

JS has what's called an event loop. It is a one dimensional array of jobs to be done. When you execute a JS program it pops in an event to the queue, and executes it in First In First Out order. It does so sequentially and goes from the top of the program to the bottom. It will pop in whatever is at line 1 to the queue, then line 2, then line 3 and so on. It will execute those instructions in that order.

So writing this:

let a = 0;
let b = a + 10;

a = b; //a == 10 as expected.

It will (and has to in order for the program to work) execute from top to bottom. It will not declare the variable b until variable a is declared, and will not reassign variable a until b has been declared. This makes sense to us because we also think sequentially.

This is nice, but sometimes we want JavaScript to do something while we wait for something else. For example, if you set a timeout with the setTimeout function.

let a = 0;
let b

setTimeout(() => {
    b = a + 10
}, 0) //Setting a timeout of 0?

a = b; //here a == undefined??...

Now look at that example and think about it. Why does setting a timeout of 0 not do the thing that we expect it to do? If we set a timeout with no wait, we expect it to happen right away, why doesn't it??

There is a simple answer to this and it is in the event loop. Javascript does thing in the order of first in first out, remember?

When it comes time for actually executing the calls in the order this happens:

    declare "a" equal to 0
    declare "b" equal to undefined

    set a timeout of 0

    declare "a" equal to the value of "b" (which is undefined)

    check if timeout is done

    now that the timeout is done, add the anonymous function that was passed inside setTimeout to the queue.
    assign to "b" the value of "a" + 10

So if we look at the above, we see that setTimeout is not blocking the rest of the program to be added to the queue. Doing things asynchronously allow for larger timeouts to happen in the background without holding up the rest of the code from executing. If the timeout would have been longer, the output would have been no different. The timeout is waiting for the timer to be done to add the item to the bottom of the queue, but not before it adds the rest of the program to the queue.

Understanding the interworkings of JS like these allows you to write more deliberate code, and makes it so the way that you make programs are more readable and take advantage of the way that the language was designed to your favor. Without it, programming JavaScript is more like the tail wagging the dog.

Hope this helps you if you are having trouble understanding async code, stay tuned for more.

What is the Dom, the Virtual Dom, and the Shadow Dom

Javier Gonzalez — Fri, 08 Jan 2021 21:37:52 +0000

There is confusion out there between these three terms. The DOM is an absolutely integral portion of modern web development, and knowing how to differentiate the three and knowing when to use these is crucial for being an effective web developer.

The DOM

So when you have markup on your page with HTML, your browser compiles all that into a Document Object Model which we abbreviate to the DOM. This is an application programming interface (API) that the browser gives you in order to be able to interact with the contents that are rendered on the page.

This sounds more complicated than it is, because at its simplest, you can throw an html file into a server and what will come out are those contents rendered, just like you wrote them. You can think of the DOM as the way that the html is presented to you when you inspect your browser window and go to the elements tab.

The Virtual Dom

The virtual DOM is like the regular DOM, except that it is held in JavaScript memory. What that means is that the virtual dom has all the properties of the regular HTML elements that you usually see on the page, except that the browser has not rendered them. You can't actually see it, but you can interact with it through JavaScript. This is how frameworks like react, vue and angular make web pages with high degrees of interactivity. If you turn off JavaScript and load a react application, you might find a single DOM node and a blank page.

This allows many different and cool things for web applications. Not having to worry about rerendering every time that something is changed, allows frameworks like react to update only the nodes which have changed when the data gets updated. These diffing algorithms are extremely powerful and make these frameworks very effective ( ...and sometimes fast!).

The Shadow Dom

The Shadow DOM is something completely different than the virtual DOM. The shadow DOM is a browser specification for building completely isolated HTML elements that are not affected by the styles that are currently affecting the rest of the page. It is one of the specifications that are aiming at making modular HTML components which can be imported into and out of different documents.

I'd say that this is not so popular in enterprise applications, but it allows for building HTML without the overhead of frameworks through the Web Component Specification. It can be tricky if you encounter it and don't know how to use it, but very helpful if you use it deliberately.

Hope that this helps you out if you are having trouble figuring out the differences.