DEV Community: Conrad Reeves

Izuku - 2D Data Visualization and Manipulation in NodeJS

Conrad Reeves — Sun, 30 Jan 2022 17:34:03 +0000

Izuku is a simple, fast, and powerful tabular data representation and manipulation library written in TypeScript. It is designed to be used to view, manipulate and debug 2D data in NodeJS applications.

The core of Izuku is the Frame class that represents a 2D array of data. It is designed to be used as a data structure for tabular data. Izuku is heavily inspired by Pandas.

Note: Izuku is not a replacement for Pandas and should not be used for data analysis. It is designed to be used for data visualization and debugging. It can, however, handle large datasets and help you understand your data better but comes at some cost in performance. Since, Pandas is based on NumPy, and NumPy is written in C, Pandas would be much faster than Izuku.

Installation

Izuku is available on npm. To install Izuku, run the following command:

npm install izuku

Showing Data

As defined above, the basic usage of Izuku is to create a Frame object and manipulate it. You can use either an 2D array, a JSON object or file, or a CSV File to create a Frame. The following example creates a Frame from a 2D array:

import { Frame } from 'izuku';
// alternatively, const { Frame } = require('izuku');

// define some column names as an array
const header = ['Name', 'Age', 'Gender', 'Country'];

// define some data as a 2D array
const data = [
  ['Arthur', 21, 'Male', 'USA'],
  ['Betty', 20, 'Female', 'Canada'],
  ['Victor', 25, 'Male'],
  ['Dodger', 30, 'Male', 'Canada'],
  ['Rayan', 21, 'Male', 'Russia'],
  ['Skitley', 29, 'Female', 'Germany'],
  ['Victoria', 89, 'Female', 'UK'],
  ['Tiger', 23, 'Male', 'India'],
  ['Killjoy', null, 'Female', 'Riot']
];

// create a frame, header is optional
const frame = new Frame(data, header);

// print the frame
frame.show();

The above code creates a frame and prints it to the console. It looks like this:

┌───────┬──────────┬─────┬────────┬─────────┐
│ Index │ Name     │ Age │ Gender │ Country │
├───────┼──────────┼─────┼────────┼─────────┤
│ 0     │ Arthur   │ 21  │ Male   │ USA     │
│ 1     │ Betty    │ 20  │ Female │ Canada  │
│ 2     │ Victor   │ 25  │ Male   │         │
│ ...   │ ...      │ ... │ ...    │ ...     │
│ 6     │ Victoria │ 89  │ Female │ UK      │
│ 7     │ Tiger    │ 23  │ Male   │ India   │
│ 8     │ Killjoy  │     │ Female │ Riot    │
└───────┴──────────┴─────┴────────┴─────────┘

Loading from JSON or CSV file

You can also load frames from JSON or CSV files. Here's how:

// if you have a JSON file, import it or you can use JSON object
import JSONData from './users.json';

const frame = new Frame().fromJSON(JSONData)

// import path for loading CSV file
import path = require('path');

// define the CSV file path
const csvPath = path.join(__dirname, 'users.csv');

const frame = new Frame().fromCSV(csvPath)

Frame Properties

There are some properties attached to the frame class. You can tap into those properties by using the dot (.) notation.

Note: Frame methods are not available on the properties. You need to console.log(propertyName) to see the property values.

`rowdata`

The rowdata property is an array of arrays that represents the data in the frame.

Note: The rowdata property is read-only. If you want to modify the data in the frame, you can use the data() method.

const rowdata = frame.rowdata;
console.log(rowdata); // prints "data" array

`columns`

The columns property is an array of strings that represents the column names in the frame.

Note: The columns property is read-only. If you want to modify the column names in the frame, you can use the header() method.

const columns = frame.columns;
console.log(columns); // prints "header" array

`size`

The size property gives the number of elements present in the frame.

Note: The size property is read-only and is automatically generated when the frame is created. Size can change if data is modified.

const size = frame.size;
console.log(size); // prints size. ex: 9

`shape`

The shape property gives the number of rows and columns present in the frame.

Note: The shape property is read-only and is automatically generated when the frame is created. Shape can change if data is modified.

const shape = frame.shape;
console.log(shape); // prints shape. ex: 9 x 4

Frame Methods

There are some methods attached to the frame class. You can tap into those methods by using the dot (.) notation. Most of the methods are chainable.

`data()`

The data() method is used to modify the data in the frame. It takes the same data argument as the constructor – which is an array of arrays.

Note: If you use data method without passing any argument, it will simply return the frame.

const data = [[...], [...], ...];
const frame = new Frame();
frame.data(data);

// modify the data
const newData = [[...], [...], [...], [...], ...];
frame.data(newData);

`header()`

The header() method is used to modify the column names in the frame. It takes the same header argument as the constructor – which is an array of strings.

Note: If you use header method without passing any argument, it will reset the column names to default header (Remember: header is optional).

// modify the header
const newHeader = [...];
frame.header(newHeader);

// Reset the header to default
frame.header();

// You can use any empty value to reset the header to default, for example:
// frame.header('');
// frame.header(null);
// frame.header(undefined);
// frame.header([]);

`column()`

The column() method is used to get the column data of a particular column. It takes the column name or the index as an argument. It can also take an array of column names or indexes as an argument to get multiple columns.

The column() method returns a new frame with extracted column data as the data of the frame. You can chain other frame methods on the returned frame.

Get a single column

// get a single column on Index 2 (Index starts from 0)
const column = frame.column(2);

// Alternatively, you can use the column name
const column = frame.column('Name');

// print the column
column.show();

Get multiple columns

// get multiple columns on Index 2 and 3 (Index starts from 0)
const columns = frame.column([2, 3]);

// Alternatively, you can use the column names
const columns = frame.column(['Name', 'Age']);

// print the columns
columns.show();

`row()`

The row() method is used to get the row data of a particular row. It takes the row index as an argument. It can also take an array of row indexes as an argument to get multiple rows.

The row() method returns a new frame with extracted row data as the data of the frame. You can chain other frame methods on the returned frame.

Get a single row

// get a single row on Index 2 (Index starts from 0)
const row = frame.row(2);

// print the row
row.show();

Get multiple rows

// get multiple rows on Index 2 and 3 (Index starts from 0)
const rows = frame.row([2, 3]);

// print the rows
rows.show();

`find()`

The find() method is used to find the rows that match the given condition. It takes a string or a number as an argument which is needed to be found in the frame. Optionally, it also takes an options object as as the second argument.

The valid options are defined below:

row: The row index to seach in. Can also be an array of row indexes.
column: The column name or index to search in. Can also be an array of column names or indexes.
strict: If true, the search will be performed on the exact value. If false, the search will be performed on the value as a substring. Default is false.

Hint: You can also combine the range() helper method to pass a range of rows or columns.

// find all the rows with value 'John' in column 'Name'
const row = frame.find('John', {column: 'Name'});

// find all the rows with value 'John' in columns 0, 1 and 2. Perform a strict search
const row = frame.find('John', {column: [0, 1, 2], strict: true});

// find all the rows with value 'John' in columns 0, 1 and 2 and rows 3, 4 and 5.
// Perform a non-strict search
const row = frame.find('John', {column: [0, 1, 2], row: [3, 4, 5], strict: false});

`sort()`

The sort() method is used to sort the rows in the frame. It takes the column name or the index as an argument. It also takes an optional ascending or descending argument to sort the rows in ascending or descending order. Default is ascending.

Note: sort() method permanently modifies the frame. You can chain other frame methods on the returned frame.

// sort the rows in the frame with column 'Name' 
frame.sort('Name');

// sort the rows in the frame with column 'Name' in descending order
frame.sort('Name', 'descending');

// sort the rows in the frame with column index 0 in ascending order
frame.sort(0, 'ascending');

`show()`

The show() method is used to print the frame. It takes no argument.

show() is a print method and it does not return a new frame and therefore it is not chainable.

// print the frame
frame.show();

You can learn more methods at the official documentation

Helper Methods

Helper methods are methods that are used to help you with some common tasks. They are not chainable. They are not required to use the frame methods.

`range()`

range() is a helper method that is used to create a an array of numbers. Here are the arguments and their default values:

Argument	Description	Default Value
`start`	The start of the range	Required
`end`	The end of the range	Required
`step`	The step size of the range	1
`remove`	An array of numbers which should not be included in the range	`undefined`

Note: The step and remove arguments are optional. If you do not provide it, step will be set to 1 and remove will be set to undefined.

Example

range(0, 10);
// [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]

range(0, 10, 2);
// [0, 2, 4, 6, 8, 10]

range(0, 10, 1, [1, 3, 5, 7, 9]);
// [0, 2, 4, 6, 8, 10]

Example with frame

// Import Frame and range
import { Frame, range } from 'izuku';

// create a frame
const frame = new Frame([[...], [...], ...]);

// Get all columns from 2 to 6
const columns = frame.column(range(2, 6));

You can learn more helper methods at the official documentation

Chaining Methods

Since, the row and column methods return a new frame, you can chain them together to get the data of multiple rows and columns. The following example shows how to get the data of multiple rows and columns.

The returned data is also a new frame and you can continue chaining other frame methods on it.

const data = [[...], [...], ...];
const header = [...];
const frame = new Frame(data, header);

// 1. get multiple rows on Index 2 and 3
// 2. then get the 'Name' and 'Age' columns of those rows
// 3. finally print the data
frame.row([2, 3]).column(['Name', 'Age']).show();

Similarly you can chain more methods if required.

So, that's Izuku. It a new library. Go, show some love by starring the library at: https://github.com/luciferreeves/izuku.js. Also, if you can, contribute to the library or open an issue if you would like to see any features.

Maintaining FakerJS - Blaver

Conrad Reeves — Sat, 08 Jan 2022 12:08:01 +0000

Warning: Self Promotion in Place, but for good reasons.

What happened?

As you all might be aware that FakerJS' developer (Marak) decided to remove all the code and take down the repository. Soon enough, people started reviving FakerJS - however the code which was propagating was a code from 6 months ago – all test cases failed and a few parts were incomplete - seemed like the code was in progress. Everyone forked the same code and published on NPM. Packages like withshepherd/faker and community-faker started getting traction.

What is Blaver? Why a new Fork?

Blaver has the same ideology of reviving and maintaining FakerJS. To be honest, I started working on Blaver before I could look at those community forks – there is no other reason to explain this. However, I noticed the incomplete code, so I detached the repository fork which I had and started fixing the library. I had to go through all (exaggerated - I meant “a lot”) of the contributors of Faker, looked at their latest fixes and fixed the code to a working state. I rolled out my own fixes too (Took me almost 100 commits, Never did so many commits in a day).

What makes it different from other community versions?

Easy to remember unique short name.
Added bigInt() function (missing from other repos) and fixed bitcoinAddress() function (as found in contributor repos).
Fixed all linting errors.
Fixed all unit test cases (0 tests were passing at the time of the fork, all tests pass now).
Remove old public API from vendor/mersenne.js.
Replaced jshint with eslint. Removed jshint specific files: jshintrc and jshintignore.
Updated all locales to include new random data and added a new Urdu (UR) locale.
FakerCloud (service used to generate random profile pictures) is also taken down. Replaced it with Pravatar. Also fixed test cases for the random pictures function.
Fixed coverage script with working coveralls code coverage.
Added GitHub Actions script to run lint and test scripts.
Update all faker instances to blaver.

Where to find it?

GitHub: https://github.com/luciferreeves/blaver

NPM: https://npmjs.com/package/blaver

Hope you like this work. I am determined to maintain this repository from now on. If you have some time, consider adding yourself to the stargazers list on the GitHub page. If you like to see any extra functionality, consider raising an Issue.

Thanks for reading this much.

Big O Notation - explained as easily as possible

Conrad Reeves — Tue, 19 Jan 2021 09:16:53 +0000

— Originally published on that computer scientist. Consider following the original blog for up-to-date articles.

If you want to learn the math involved with the Big O, read Analysing Algorithms: Worst Case Running Time.

Data Structures and Algorithms is about solving problems efficiently. A bad programmer solves their problems inefficiently and a really bad programmer doesn't even know why their solution is inefficient. So, the question is, How do you rank an algorithm's efficiency?

The simple answer to that question is the Big O Notation. How does that work? Let me explain!

Say you wrote a function which goes through every number in a list and adds it to a total_sum variable.

If you consider "addition" to be 1 operation then running this function on a list of 10 numbers will cost 10 operations, running it on a list of 20 numbers costs 20 operations and similarly running it on a list of n numbers costs the length of list (n) operations.

Now let's assume you wrote another function that would return the first number in a list.

Now, no matter how large this list is, this function will never cost more than one operation. Fairly, these two algorithms have different time complexity or relationship between growth of input size and growth of operations executed. We communicate these time complexities using Big O Notation.

Big O Notation is a mathematical notation used to classify algorithms according to how their run time or space requirements grow as the input size grows.

Referring to the complexities as 'n', common complexities (ranked from good to bad) are:

Constant - O(1)
Logarithmic O(log n)
Linear - O(n)
n log n - O(n log n)
Quadratic - O(n²)
Exponential - O(2ⁿ)
Factorial - O(n!)

Our first algorithm runs in O(n), meaning its operations grew in a linear relationship with the input size - in this case, the amount of numbers in the list. Our second algorithm is not dependent on the input size at all - so it runs in constant time.
Let's take a look at how many operations a program has to execute in function with an input size of n = 5 vs n = 50.

	n = 5	n = 50
O(1)	1	1
O(log n)	4	6
O(n)	5	50
O(n log n)	20	300
O(n²)	25	2500
O(2ⁿ)	32	1125899906842624
O(n!)	120	3.0414093e+64

It might not matter when the input is small, but this gap gets very dramatic as the input size increases.

If n were 10000, a function that runs in log(n) would only take 14 operations and a function that runs in n! would set your computer on fire!

For Big O Notation, we drop constants so O(10.n) and O(n/10) are both equivalent to O(n) because the graph is still linear.

Big O Notation is also used for space complexity, which works the same way - how much space an algorithm uses as n grows or relationship between growth of input size and growth of space needed.

So, yeah! This has been the simplest possible explanation of the Big O Notation from my side and I hope you enjoyed reading this. If you found this information helpful, share it with your friends on different social media platforms and consider clicking those like buttons up there, it really motivates me to write more.

And If you REALLY liked the article, consider buying me a coffee 😊. If you have any suggestions or feedback, feel free to let me know that in the comments.

Happy Programming!