DEV Community: Lyndsi Kay Williams

Apache Superset: Installing locally is easy using the makefile

Lyndsi Kay Williams — Sun, 20 Aug 2023 18:01:01 +0000

Are you interested in trying out Superset, but you're intimidated by the local setup process? Worry not! Superset needs some initial setup to install locally, but I've got a streamlined way to get started - using the makefile! This file contains a set of scripts to simplify the setup process.

This setup is for Mac OS X, for other OS setups see: https://superset.apache.org/docs/installation/installing-superset-from-scratch/

To start, fork and download Superset from Github: https://github.com/apache/superset

You can run Superset directly without forking it, but you won't be able to create pull requests unless you're using your own forked version.

Prerequisites

Superset has some prerequisites in order to run properly. Let's start with dependencies:

Make sure to update your machine to the latest version of Mac OS X. After updating, install the latest version of XCode command line tools:

xcode-select --install

Going forward, we'll be using Homebrew to install dependencies: https://brew.sh/

Node: Currently, Superset works best with Node.js version 16 and npm version 7. If you've got a preferred version of node, I suggest using Node Version Manager (NVM).

See here for full NVM installation instructions

nvm install 16

Python: Currently, Superset works best with Python version 3.9. We'll be using a Python virtual environment, so for now just install the Python version:

brew install python@3.9

Make sure to update pip and setuptools as well:

pip install --upgrade setuptools pip

Almost done, now we need the rest of the dependencies:

brew install readline pkg-config libffi openssl mysql postgresql@14

Set LDFLAGS and CFLAGS to allow certain Python packages to build properly. You can export the variables with these commands:

export LDFLAGS="-L$(brew --prefix openssl)/lib"
export CFLAGS="-I$(brew --prefix openssl)/include"

You will need to set a secret key in your superset_config.py file for security reasons. Open superset in your favorite code editor and create a file called superset_config.py in the root folder. In this file, set your secret key like this:

SECRET_KEY='insertSecurePasswordHere'

Python Virtual Environment

Now we can set up our Python virtual environment. Start by installing virtualenv:

pip install virtualenv

cd into wherever you've locally cloned your forked instance of Superset. Create a virtual environment with Python 3.9:

// Create a venv directory
python3.9 -m venv venv

// Open the virtual environment
. venv/bin/activate

You will need two terminal windows to run Superset, I like to use iTerm2's horizontal window stacking to keep my terminal windows together: https://iterm2.com/

Now it's time for the Makefile magic ✨

Make sure you're at the root file of Superset and in a virtual environment. Run this command to install Superset:

make install

That's it, you're installing Superset! This script will run for a bit, bring a snack.

Once the installation is complete, it's time to get things running. Boot up the back end with:

make flask-app

In a separate terminal window, once again make sure you're at the root file of Superset and in a virtual environment. Boot up the front end with:

make node-app

Once webpack finishes loading the app, go to localhost:9000 in your browser. You'll see a login screen, log in with these credentials:

Username: admin
Password: general

You're in! Welcome to Superset, enjoy your experience and feel free to contribute. It's open source!

Real DOM, Virtual DOM, Shadow DOM, What's the Difference?

Lyndsi Kay Williams — Sun, 16 Jul 2023 14:40:20 +0000

Real DOMs and virtual DOMs and shadow DOMs, oh my! Let's take a dive to see how they all work together to create a clean, performant Document Object Model.

The DOM (Document Object Model) is exactly as it states. The HTML tree of a website is represented by an object called document. In this object there is a model of the HTML tree's elements, conveniently accessible by object dot notation.

document.head will return the <head></head> section of the tree, document.body will return the <body></body> section of the tree, and so on. You can also use this dot notation to manipulate the DOM. For example: document.body.style.background = ‘red’ will change the body’s background color to red. You can see more about the DOM API in Mozilla's web docs.

This DOM is referred to as the page's real DOM. The real DOM, by itself, is only able to update the entire DOM simultaneously every time there is a change to the DOM. This makes it very slow and expensive to make updates to the page. That's where the virtual DOM comes to save the day!

The virtual DOM is a virtual representation of the real DOM. This virtual DOM is kept in memory and synced with the real DOM. React compiles the real DOM into Javascript, which is the first step in creating more performant updates. The virtual DOM then makes a copy of itself (let's call it virtual DOM 2). When an update is made on the page, it is first applied to virtual DOM 2. React then compares virtual DOM 2 to the original virtual DOM, an exact copy of the real DOM. React uses this comparison to quickly detect where the real DOM needs to be updated and updates only those elements instead of the entire DOM. This is where the magic happens, it is much quicker and less expensive to update only what needs to be updated.

If you want to see the virtual DOM in action, you can see a visual representation with the "Paint flashing" feature in Google Chrome's inspect tool:

Check that box and then play with the page, any DOM changes will be highlighted with a green box.

Last but not least, we have the elusive shadow DOM. This allows hidden DOM trees to be attached to elements in the regular DOM tree. Custom elements can be created with the Web API, these are controlled solely by the shadow DOM. It's important to remember that the real DOM and the shadow DOM are completely different realms. Changes made to elements in the real DOM will not apply to elements in the shadow DOM, and vice versa.

If you're sleuthing through HTML elements in the inspect tool and don't see something listed when you're looking right at it on the page, it's probably in the shadow DOM. The best example is video players. Visually on the page you can see the internal UI: the play button, skip button, share button, etc. but you don't see it in the real DOM. It's being controlled by the shadow DOM.

Elusive as it may be, you can actually reveal the shadow DOM if you need it! In Google Chrome's inspect tool: Go to the settings, select Preferences and go to the Elements section. Select "Show user agent shadow DOM" to show any shadow DOMs alongside the real DOM in the Elements tab of the inspect tool.

That's all three! Hopefully this blog has brought you a better understanding of the differences among the real DOM, the virtual DOM, and the shadow DOM. Let me know what you think in the comments!

React hooks: An advanced look at converting useState to useReducer

Lyndsi Kay Williams — Wed, 10 May 2023 15:45:09 +0000

In my previous blog, I gave a basic demonstration on how to manage state with useReducer instead of useState. In this blog, I'll dive into a more complex example.

When managing state for a form, things can get complex quickly. Take this form, for example:

To manage the form's state locally with useState, it would look something like this:



const [name, setName] = useState('')
const [title, setTitle] = useState('')
const [favoriteNumber, setFavoriteNumber] = useState('')
const [favoriteColor, setFavoriteColor] = useState('')

This might seem manageable enough for now - but if the form needs to grow, one useState per input will turn your code into spaghetti before you know it! Let's take a look at how we can make state management more scaleable with useReducer using the same steps shown in my previous blog:

Define an initial state and an action type.
Replace the useState hook with the useReducer hook.
Create a reducer function that takes a state and an action and returns a new state.
Update the component to use the new state and dispatch functions returned by the useReducer hook, which should have the reducer and initial state passed in.

Following the steps above, we can convert our useState state to useReducer state.

1. Define an initial state and an action type:



const initialState = {
  name: 'Default name',
  title: 'Default title',
  favorite_number: '3',
  favorite_primary_color: 'red',
};

const INPUT_CHANGE = 'inputChange';

2. Replace the `useState` hook with the `useReducer` hook:



import { useReducer } from 'react';

const FormComponent = () => {
  const [formData, setFormData] = useReducer(formReducer, initialState);

  // ...
}

3. Create a reducer function that takes a state and an action and returns a new state:



const formReducer = (state, action) => {
  const stateCopy = { ...(state || {}) };

  switch(action.type) {
    case INPUT_CHANGE:
      return {
        ...stateCopy,
        [action.payload.name]: action.payload.value
      }

    default:
      return null;
  }
}

4. Update the component to use the new state and dispatch functions returned by the `useReducer` hook, which should have the reducer and initial state passed in:



const FormComponent = () => {
  const [formData, setFormData] = useReducer(formReducer, initialState);

  const internalOnchange = (type, payload) => {
    setFormData({ type, payload })
  };

  // ...

  return (
    <form>
      <input
        type="text"
        id="name"
        name="name"
        onChange={event => {
          internalOnchange(INPUT_CHANGE, {
            name: event.target.name,
            value: event.target.value,
          });
        }}
      />

    {/* ... */}

    </form>
  );
}

Step 4 is a little more complex in this example. In order to pass the type and payload to the reducer cleanly, I've implemented an internalOnchange helper function. Now the internalOnchange can be passed into any input on the form, as shown on the name input above.

As long as the name attribute of the input matches the data you want to change, that value in state will be changed. Now the form can have unlimited inputs with no need to change the state management functionality!

...unless the form gets even more complex?? Not a problem, you can add another action to your formReducer to manage any specialized inputs. To see this form as a completed project, check it out here on Github.

I've created the example form explained above - it changes state independently by action, but I've also included an option to change the data as a whole by manipulating the entire JSON directly. This is done by passing the entire changed JSON as the payload here and using an updateData action, created here. This project will look a little different than the blog example since it's done in TypeScript, I wanted to keep things simple in this blog so I explained it in JavaScript.

I hope this blog clearly explained using useReducer to handle state instead of useState. If you have any questions or comments, I'd love to hear what you've got!

React hooks: How to convert useState to useReducer

Lyndsi Kay Williams — Wed, 22 Feb 2023 01:43:03 +0000

useState and useReducer are both React hooks that allow you to manage state in your components. useState is a simpler hook that allows you to manage a single piece of state, while useReducer is more powerful and allows you to manage more complex state that may have multiple values and require more sophisticated updates.

To convert from useState to useReducer, you can follow these general steps:

Define an initial state and an action type.
Replace the useState hook with the useReducer hook.
Create a reducer function that takes a state and an action and returns a new state.
Update the component to use the new state and dispatch functions returned by the useReducer hook, which should have the reducer and initial state passed in.

Here is a basic example of how to convert a component that uses useState to manage a single piece of state. Start with a Counter component:

import React, { useState } from 'react';

const Counter = () => {
  const [count, setCount] = useState(0);

  const increment = () => {
    setCount(count + 1);
  };

  return (
    <>
      <p>Count: {count}</p>
      <button onClick={increment}>Increment</button>
    </>
  );
}

export default Counter;

To convert this to use useReducer, we can follow the steps outlined above:

1. Define an initial state and an action type:

const initialState = { count: 0 };
const INCREMENT = 'increment';

2. Replace the `useState` hook with a `useReducer` hook:

import React, { useReducer } from 'react';

const Counter = () => {
  const [state, dispatch] = useReducer(reducer, initialState);

  // ...
}

3. Create a reducer function that takes a state and an action and returns a new state:

const reducer = (state, action) => {
  switch (action.type) {
    case INCREMENT:
      return { count: state.count + 1 };
    default:
      throw new Error();
  }
}

4. Update the component to use the new state and dispatch functions returned by the `useReducer` hook, which should have the reducer and initial state passed in:

const Counter = () => {
  const [state, dispatch] = useReducer(reducer, initialState);

  const increment = () => {
    dispatch({ type: INCREMENT });
  };

  return (
    <>
      <p>Count: {state.count}</p>
      <button onClick={increment}>Increment</button>
    </>
  );
}

In step 4, the dispatch function is used to send an action to the reducer function. The reducer function then returns a new state based on the action type. The component is also updated to use state.count instead of the count variable previously returned by useState.

In this simple example, useReducer doesn't bring many benefits over useState. But in general, as your components grow in complexity, useReducer provides a more powerful and flexible way to manage state - especially when you have more complex state that require multiple values and more sophisticated updates.

Apache Superset: Testing and Enzyme to RTL conversion

Lyndsi Kay Williams — Tue, 31 May 2022 17:56:53 +0000

Superset uses Jest and React Testing Library (RTL) to write unit and integration tests. In the past we used Enzyme, but now that we're currently converting all of our class components to functional components, Enzyme cannot support our testing needs. Since RTL is better for testing functional components, we're converting all of our test files to RTL. This can be quite a learning curve - I've gone through a lot of the process so I'd like to share what I've learned so far.

First and foremost: We have a Testing Guidelines and Best Practices document that outlines all the code styles we adhere to in our tests. Please use this document as reference when writing your tests.

The main thing you want to keep in mind is the different approach each testing library uses in testing:

Enzyme is code-focused testing. It tests things in the "back" of the code that the user never sees, such as the props of a component.
RTL is user-focused testing. All of your tests will rely on things the user can see and do.
- For example: If there were an Enzyme test that checks the props of a component, you would look for where those props show visually on the component and capture it with RTL.
- Except for certain things like API mocking, Superset uses fetch-mock for that. Here's a link to the docs, if you'd like to learn more about how to mock API calls in RTL. You can also look for other already-existing instances of fetch-mock in the codebase to see how it's used in Superset.

The test environment in Superset is getting healthier and easier to work with by the day, and I hope this will inspire you to help contribute to the cause! To wrap this up, I would like to leave you with some good RTL/testing resources:

In your tests, use these commands to help with debugging:
- screen.debug()
  - This will show you the rendered component in the console when you run the test. You can pass in a captured element if you want to see just that element instead of the whole rendered component.
- screen.logTestingPlaygroundURL()
  - This will give you a link in the console when you run your test. Click this link and it will open an instance of the testing playground displaying your rendered component (or a captured element passed into the function) in the browser. Check out this link for an example of what that looks like, this is a really cool feature 😁 It would look like this in the code:

const button = screen.getByRole('button');
screen.debug(button);
screen.logTestingPlaygroundURL(button);

I converted 11 files in this pull request, feel free to use it as a reference for conversions
The RTL docs have a good guide for converting from Enzyme
RTL cheatsheet
Jest matchers
jest-dom matchers
Superset has a very handy helper file for RTL with supercharged functionality for our specific needs.

DEV Community: Lyndsi Kay Williams

Apache Superset: Installing locally is easy using the makefile

Prerequisites

Python Virtual Environment

Now it's time for the Makefile magic ✨

Real DOM, Virtual DOM, Shadow DOM, What's the Difference?

React hooks: An advanced look at converting useState to useReducer

1. Define an initial state and an action type:

2. Replace the useState hook with the useReducer hook:

3. Create a reducer function that takes a state and an action and returns a new state:

4. Update the component to use the new state and dispatch functions returned by the useReducer hook, which should have the reducer and initial state passed in:

React hooks: How to convert useState to useReducer

1. Define an initial state and an action type:

2. Replace the useState hook with a useReducer hook:

3. Create a reducer function that takes a state and an action and returns a new state:

4. Update the component to use the new state and dispatch functions returned by the useReducer hook, which should have the reducer and initial state passed in:

Apache Superset: Testing and Enzyme to RTL conversion

2. Replace the `useState` hook with the `useReducer` hook:

4. Update the component to use the new state and dispatch functions returned by the `useReducer` hook, which should have the reducer and initial state passed in:

2. Replace the `useState` hook with a `useReducer` hook:

4. Update the component to use the new state and dispatch functions returned by the `useReducer` hook, which should have the reducer and initial state passed in: