DEV Community: William Lyon

Introducing The Spatial Cypher Cheat Sheet

William Lyon — Wed, 28 Jun 2023 22:18:00 +0000

A Resource For Working With Geospatial Data In Neo4j

In this post we explore some techniques for working with geospatial data in Neo4j. We will cover some basic spatial Cypher functions, spatial search, routing algorithms, and different methods of importing geospatial data into Neo4j.

I recently went through the examples in the Spatial Cypher Cheat Sheet in an episode of the Neo4j Livestream, you can watch the recording below:

The first page of the Spatial Cypher Cheat Sheet introduces Cypher and the property graph data model, the spatial types available in the Neo4j database, as well as some of the spatial functions available in Cypher. We also touch on importing geospatial data into Neo4j (from CSV and GeoJSON) as well as some of the path-finding algorithms (breadth-first search, Dijkstra’s Algorithm, and A*).

Page 2 of the Spatial Cypher Cheat Sheet covers using Neo4j with Python. First, using the Neo4j Python driver to query data from Neo4j to build a GeoDataFrame. We then explore using the OSMNx Python package for fetching data from OpenStreetMap to load a road network into Neo4j.

You can download the PDF version of the Spatial Cypher Cheat Sheet, or read on for the same content in the rest of this blog post.

Introduction To Geospatial Cypher Functions With Neo4j

Intro To Cypher & The Property Graph Data Model
Spatial Point Type
Spatial Cypher Functions
Data Import
Routing With Path Finding Algorithms

Using Neo4j With Python

The Neo4j Python Driver
Creating A GeoDataFrame From Data Stored In Neo4j
Working With OpenStreetMap Data
Loading A Road Network With OSMNx
Analyzing and Visualizing Road Networks With Neo4j Bloom And Graph Data Science

Introduction To Geospatial Cypher Functions With Neo4j

Intro To Cypher And The Property Graph Data Model

Neo4j is a database management system (DBMS) that uses the property graph data model which is composed of nodes, relationships, and properties to model, store, and query data as a graph. Nodes can have one or more labels, relationships have a single type and direction. Key-value pair properties can be stored on nodes and relationships.

The Cypher query language is used to query data and interact with Neo4j. Cypher is a declarative query language that uses ASCII-art like syntax to define graph patterns that form the basis of most query operations. Nodes are defined with parenthesis, relationships with square brackets, and can be combined to create complex graph patterns. Common Cypher commands are MATCH (find where the graph pattern exists), CREATE (add data to the database using the specified graph pattern), and RETURN (return a subset of the data as a result of a traversal through the graph.

MATCH p=(sfo:Airport {iata: "SFO"})-[:FLIGHT_TO*2]->(rsw:Airport {iata: "RSW"}) RETURN p

Spatial Point Type

Neo4j supports 2D or 3D geographic (WGS84) or cartesian coordinate reference system (CRS) points. Here we create a point by specifying latitude/longitude, WGS84 is assumed when using latitude/longitude:

RETURN point( {latitude:37.62245, longitude:-122.383989} )

Point data can be stored as properties on nodes or relationships. Here we create an Airport node and set its location as a point.

CREATE (a:Airport)
SET a.iata = "SFO",
a.location = point( {latitude:37.62245, longitude:-122.383989})
RETURN a

Database indexes are used to speed up search performance. Here we create a database index on the location property for Airport nodes. This will help us find airports faster when searching for airports by location (radius distance or bounding box spatial search).

CREATE POINT INDEX airportIndex FOR (a:Airport) ON (a.location)

Spatial Cypher Functions

Radius Distance Search

To find nodes close to a point or other nodes in the graph we can use the point.distance() function to perform a radius distance search

MATCH (a:Airport)
WHERE point.distance(
  a.location,
  point({latitude:37.55948, longitude:-122.32544})) < 20000
RETURN a

Within Bounding Box

To search for nodes within a bounding box we can use the point.withinBBox() function.

MATCH (a:Airport)
WHERE point.withinBBox(
  a.location,
  point({longitude:-122.325447, latitude: 37.55948 }),
  point({longitude:-122.314675 , latitude: 37.563596}))
RETURN a

See also my other blog post that goes into a bit more detail on spatial search functionality with Neo4j, including point in polygon: Spatial Search Functionality With Neo4j

Geocoding

To geocode a location description into latitude, longitude location we can use the apoc.spatial.geocode() procedure. By default this procedure uses the Nominatim geocoding API but can be configured to use other geocoding services, such as Google Cloud.

CALL apoc.spatial.geocode('SFO Airport') YIELD location
---------------------------------------------------------------
{
  "description": "San Francisco International Airport, 780, South Airport Boulevard, South San Francisco, San Mateo County, CAL Fire Northern Region, California, 94128, United States",
  "longitude": -122.38398938548363,
  "latitude": 37.622451999999996,
}

Data Import

We can use Cypher to import data into Neo4j from formats such as CSV and JSON, including GeoJSON.

CSV

Using the LOAD CSV Cypher command to create an airport routing graph.

1 - Create a constraint on the field that identies uniquesness, in this case Airport IATA code. This ensures we won’t create duplicate airports but also creates a database index to improve performance of our data import steps below.

CREATE CONSTRAINT FOR (a:Airport) REQUIRE a.iata IS UNIQUE;

2 - Create Airport nodes, storing their location, name, IATA code, etc as node properties.

LOAD CSV WITH HEADERS
FROM "https://cdn.neo4jlabs.com/data/flights/airports.csv"
AS row
MERGE (a:Airport {iata: row.IATA_CODE})
ON CREATE SET a.city = row.CITY,
              a.name = row.AIRPORT,
              a.state = row.STATE,
              a.country = row.country,
              a.location =
                point({ latitude: toFloat(row.LATITUDE),
                       longitude: toFloat(row.LONGITUDE)
                });

3 - Create FLIGHT_TO relationships connecting airports with a connecting flight. Increment the num_flights counter variable to keep track of the number of flights between airports per year.

:auto
LOAD CSV WITH HEADERS
FROM "https://cdn.neo4jlabs.com/data/flights/flights.csv" AS row
CALL {
  WITH row
  MATCH (origin:Airport {iata: row.ORIGIN_AIRPORT})
  MATCH (dest:Airport {iata: row.DESTINATION_AIRPORT})
  MERGE (origin)-[f:FLIGHT_TO]->(dest)
    ON CREATE SET
      f.num_flights = 0, f.distance = toInteger(row.DISTANCE)
    ON MATCH SET
      f.num_flights = f.num_flights + 1
} IN TRANSACTIONS OF 50000 ROWS;

GeoJSON

We can also store arrays of Points to represent complex geometries like lines and polygons, for example to represent land parcels.

CALL apoc.load.json('https://cdn.neo4jlabs.com/data/landgraph/parcels.geojson')
YIELD value
UNWIND value.features AS feature
CREATE (p:Parcel) SET
  p.coordinates = [coord IN feature.geometry.coordinates[0] | point({latitude: coord[1], longitude: coord[0]})]
  p += feature.properties;

Routing With Path Finding Algorithms

Shortest Path

The shortestPath function performs a binary breadth-first search to find the shortest path between nodes in the graph.

MATCH p = shortestPath(
    (:Airport {iata: "SFO"})-[:FLIGHT_TO*..10]->(:Airport {iata: "RSW"})
) RETURN p

Shortest Weighted Path

Often we want to consider the shortest weighted path taking into account distance, time or some other cost stored as relationship properties. Dijkstra and A* are two algorithms that take relationship (or edge) weights into account when calculating the shortest path.

Dijkstra's Algorithm

Dijkstra’s algorithm is similar to a breadth-first search, but takes into account relationship properties (distance) and prioritizes exploring low-cost routes first using a priority queue.

MATCH (origin:Airport {iata: "SFO"})
MATCH (dest:Airport {iata: "RSW"})
CALL
  apoc.algo.dijkstra(
    origin,
    dest,
    "FLIGHT_TO",
    "distance"
  )
YIELD path, weight
UNWIND nodes(path) AS n
RETURN {
  airport: n.iata,
  lat: n.location.latitude,
  lng: n.location.longitude
} AS route

A* Algorithm

The A* algorithm adds a heuristic function to choose which paths to explore. In our case the heuristic is the distance to the final destination.

MATCH (origin:Airport {iata: "SFO"})
MATCH (dest:Airport {iata: "RSW"})
CALL
  apoc.algo.aStarConfig(
    origin,
    dest,
    "FLIGHT_TO",
    {
     pointPropName: "location",
     weight: "distance"
    }
  )
YIELD weight, path
RETURN weight, path

There are additional path-finding algorithms available in Neo4j's Graph Data Science Library.

Using Neo4j With Python For Geospatial Data

The Neo4j Python Driver

In this section we’ll use the Neo4j Python Driver to create a GeoDataFrame of our flight data. We’ll also compute weighted degree centrality so we can plot airport size relative to their “importance” in the US airline network. The Neo4j Python Driver can be installed with:

pip install neo4j

Creating A GeoDataFrame From Data Stored In Neo4j

First we import the neo4j Python package, define our connection credentials for our Neo4j instance (here we are using a local Neo4j instance), and create a driver instance.

import neo4j

NEO4J_URI           = "neo4j://localhost:7689"
NEO4J_USER          = "neo4j"
NEO4J_PASSWORD      = "letmeinnow"
NEO4J_DATABASE      = "neo4j"

driver = neo4j.GraphDatabase.driver(
  NEO4J_URI, auth=(NEO4J_USER, NEO4J_PASSWORD)
)

Next, we define a Cypher query to fetch data from Neo4j. In addition to fetching each flight between airports we compute weighted degree centrality, a measure of each node's importance in the network, the sum of all relationship weights connected to a given node, in this case the number of flights per year for each airport.

We also return the geometry of our origin and destination airports, and the flight route as Well Known Text (WKT). POINT for the airports and LINESTRING for the flight route. We’ll parse this WKT when defining the geometry in our GeoDataFrame.

AIRPORT_QUERY = """
  MATCH (origin:Airport)-[f:FLIGHT_TO]->(dest:Airport)
  CALL {
   WITH origin
   MATCH (origin)-[f:FLIGHT_TO]-()
   RETURN sum(f.num_flights) AS origin_centrality
  }
  CALL {
   WITH dest
   MATCH (dest)-[f:FLIGHT_TO]-()
   RETURN sum(f.num_flights) AS dest_centrality
  }
  RETURN {
    origin_wkt:
      "POINT (" + origin.location.longitude + " " + origin.location.latitude + ")",
    origin_iata: origin.iata,
    origin_city: origin.city,
    origin_centrality: origin_centrality,
    dest_centrality: dest_centrality,
    dest_wkt:
      "POINT (" + dest.location.longitude + " " + dest.location.latitude + ")",
    dest_iata: dest.iata,
    dest_city: dest.city,
    distance: f.distance,
    num_flights: f.num_flights,
    geometry:
      "LINESTRING (" + origin.location.longitude + " " + origin.location.latitude + ","
      + dest.location.longitude + " " + dest.location.latitude + ")"
    }
    """

Next we define a Python function to execute our Cypher query and process the results into a GeoPandas GeoDataFrame. The Neo4j Python driver has a .to_df() method which will convert a Neo4j result set to a Pandas DataFrame. Note that we parse the WKT columns into GeoSeries and convert the pandas DataFrame into a GeoPandas GeoDataFrame.

def get_airport(tx):
    results = tx.run(AIRPORT_QUERY)
    df = results.to_df(expand=True)
    df.columns =
     ['origin_city','origin_wkt', 'dest_city', 'dest_wkt', 'origin_centrality', 'distance', 'origin_iata',
      'geometry','num_flights', 'dest_centrality', 'dest_iata']
    df['geometry'] = geopandas.GeoSeries.from_wkt(df['geometry'])
    df['origin_wkt'] = geopandas.GeoSeries.from_wkt(df['origin_wkt'])
    df['dest_wkt'] = geopandas.GeoSeries.from_wkt(df['dest_wkt'])
    gdf = geopandas.GeoDataFrame(df, geometry='geometry')
    return gdf

with driver.session(database=NEO4J_DATABASE) as session:
    airport_df = session.execute_read(get_airport)

We now have a GeoDataFrame where each row is a flight route between two airports. We can plot the airport and routes, using the centrality metric to size airport nodes: more important airports should be larger.

world = geopandas.read_file(
 geopandas.datasets.get_path('naturalearth_lowres')
)

ax = world[world.continent == 'North America']
 .plot(color='white', edgecolor='black')

flights_gdf = flights_gdf.set_geometry("origin_wkt")
flights_gdf.plot(ax=ax, markersize='origin_centrality')

flights_gdf = flights_gdf.set_geometry("geometry")
flights_gdf.plot(ax=ax, markersize=0.1, linewidth=0.01)

Working With OpenStreetMap Data

In this section we will import data from OpenStreetMap into Neo4j using the OSMNx Python package. Below is the property graph data model we will use to model the road network of Boston.

pip install osmnx

Loading A Road Network With OSMNx

import osmnx as ox

G = ox.graph_from_place("Boston, MA, USA", network_type="drive")
fig, ax = ox.plot_graph(G)

gdf_nodes, gdf_relationships = ox.graph_to_gdfs(G)
gdf_nodes.reset_index(inplace=True)
gdf_relationships.reset_index(inplace=True)

Here is our nodes GeoDataFrame. Each row represents an intersection in the Boston road network:

Here is our relationships GeoDataFrame. Each row represents a road segment connecting two intersections.

We'll define a Cypher query to add intersection nodes from the nodes GeoDataFrame and add road segments from the relationships GeoDataFrame connecting intersection nodes.

First let's create a constraint to ensure we don't have duplicate Intersection nodes (this will also create a node index to improve lookups during import):

CREATE CONSTRAINT FOR (i:Intersection) REQUIRE i.osmid IS UNIQUE

We can also create an index on the osmid property of the ROAD_SEGMENT relationship to improve import performance:

CREATE INDEX FOR ()-[r:ROAD_SEGMENT]-() ON r.osmid

Because our GeoDataFrames can be very large we break them up into batches to avoid sending too much data to the database at once.

node_query = '''
    UNWIND $rows AS row
    WITH row WHERE row.osmid IS NOT NULL
    MERGE (i:Intersection {osmid: row.osmid})
        SET i.location =
         point({latitude: row.y, longitude: row.x }),
            i.ref = row.ref,
            i.highway = row.highway,
            i.street_count = toInteger(row.street_count)
    RETURN COUNT(*) as total
    '''

rels_query = '''
    UNWIND $rows AS road
    MATCH (u:Intersection {osmid: road.u})
    MATCH (v:Intersection {osmid: road.v})
    MERGE (u)-[r:ROAD_SEGMENT {osmid: road.osmid}]->(v)
        SET r.oneway = road.oneway,
            r.lanes = road.lanes,
            r.ref = road.ref,
            r.name = road.name,
            r.highway = road.highway,
            r.max_speed = road.maxspeed,
            r.length = toFloat(road.length)
    RETURN COUNT(*) AS total
    '''

def insert_data(tx, query, rows, batch_size=1000):
    total = 0
    batch = 0

    while batch * batch_size < len(rows):
        results = tx.run(query, parameters = {
          'rows':
            rows[batch * batch_size: (batch + 1) * batch_size]
            .to_dict('records')
        }).data()
        print(results)
        total += results[0]['total']
        batch += 1

with driver.session() as session:
    session.execute_write(insert_data, node_query, gdf_nodes.drop(columns=['geometry']))
    session.execute_write(insert_data, rels_query, gdf_relationships.drop(columns=['geometry']))

In this post we introduced some of the spatial functionality natively supported by Neo4j including the point type and related Cypher functions and demonstrated how to accomplish various spatial search operations as well as a brief look at routing with graph algorithms.

Resources

10 Things We Learned In Full Stack GraphQL Book Club

William Lyon — Mon, 27 Feb 2023 15:43:00 +0000

My colleague Alex and I just wrapped up the Full Stack GraphQL Book Club livestream series so I thought now would be a good time to share some of the main takeaways and things we learned about building full stack applications with GraphQL.

When my book Full Stack GraphQL Applications was published I thought it would be fun to organize a book club of sorts - a series of livestream episodes where each episode we review a chapter of the book and work through the end of chapter exercises.

Full Stack GraphQL Applications is all about how to build web applications using GraphQL, React, Apollo, and Neo4j Database - the so called GRANDstack. The book is designed to show full stack developers how they can leverage the benefits of GraphQL including GraphQL database integrations like the Neo4j GraphQL Library, cloud services like Netlify, AWS Lambda, Auth0, and Neo4j AuraDB.

Below for each chapter we'll take a look at the major points from each chapter plus one interesting thing we learned during the livestream.

Chapter 1: What Is Full Stack GraphQL?

Chapter one introduces a mental model of full stack GraphQL. Using an example of a movies search web application we explore the components of a full stack GraphQL application by tracing a GraphQL request from client to database and back.

What we learned during book club
There isn't just one right set of components that make up a Full Stack GraphQL application. For example, we could be building a native iOS app written in Swift that leverages the Swift GraphQL client instead of a React application or perhaps using GraphQL Yoga instead of Apollo Server - it's more important to understand the concepts of how the pieces fit together and then you'll be better able to understand the tradeoffs of choosing different components of the stack.

Chapter 2: Graph Thinking With GraphQL

The typical approach to building GraphQL APIs starts with writing GraphQL type definitions that define the data available in the API. Then we create resolver functions with the logic for interacting with our data layer to "resolve" our data.

What we learned during book club
The N+1 query problem becomes apparent for all but the simplest GraphQL APIs. Multiple round trip requests to the data layer for each GraphQL request can become a big performance bottleneck. Fortunately, GraphQL database integrations like the Neo4j GraphQL Library address the N+1 query problem by generating a single database query for each GraphQL operation.

Chapter 3: Graphs In The Database

Graph databases like Neo4j allow developers to model, store, and query data as a graph: node and relationships that connect nodes. We use the Cypher query language for working with data in Neo4j. Cypher is all about graph pattern matching.

What we learned during book club
The Cypher query language allows us to define complex graph patterns using ASCII-art like notation representing paths in the graph.

Chapter 4: The Neo4j GraphQL Library

The Neo4j GraphQL Library allow us to build fully functional GraphQL APIs backed by the Neo4j graph database by generating database queries from arbitrary GraphQL requests.

What we learned during book club
The most powerful feature of the Neo4j GraphQL Library is the @cypher schema directive for adding custom logic to our GraphQL API using the Cypher query language.

Chapter 5: Building User Interfaces With React

React.js a JavaScript library for building user interfaces. Starting with the create-react-app CLI we see how to create and compose React components to build user interfaces and how to work with data in a React app.

What we learned during book club
Understanding how to work with state variables is an important concept in React. State variables are updated using "hooks" - or functions for hooking into React state and lifecycle features.

Chapter 6: Client Side GraphQL With Apollo Client

The React integration of Apollo Client introduces a handful of React "hook" functions for interacting with our GraphQL. The useQuery and useMutation functions are the most common React hooks for querying our GraphQL API.

What we learned during book club
We can also use Apollo Client to manage local application state with the @client GraphQL query directive. This can simplify data management code in our React application by using Apollo Client and the GraphQL schema as the single source of application data and state.

Chapter 7: Adding Authentication and Authorization

We can declaratively define authorization rules with the @auth directive when using the Neo4j GraphQL Library. We use JSON Web Tokens (JWTs) to encode user identity and implement sign-in functionality with the Auth0 React SDK.

What we learned during book club
JSON Web Token (JWT) is a standard for identity and access management that means we can implement user auth flow ourselves or choose to integrate with a managed auth service like Auth0.

Chapter 8: Deploying Our Full Stack GraphQL Application

Evaluating managed cloud services and how they work together to choose the best deployment options for our full stack GraphQL application. We use Netlify, GitHub, Auth0, AWS Lambda and Neo4j AuraDB.

What we learned during book club
Be sure to evaluate deployment options from the perspective of the developer as well as the user. We can deploy our GraphQL API as a serverless function. The Netlify platform can integrate with GitHub to deploy preview builds. We also learn about graph data visualization with Neo4j Bloom and Neo4j AuraDB.

Chapter 9: Advanced GraphQL Considerations

GraphQL abstract types (unions and interfaces), Relay-style pagination, and working with relationship properties.

What we learned during book club
Union types can be useful for representing search results when the concrete types do not necessarily share any common fields.

Takeaway #10: Bonus: Low Code GraphQL With The Neo4j GraphQL Toolbox

Being able to focus on the key parts of designing your GraphQL API can be a huge advantage for prototyping and getting new projects up and running quickly. The Neo4j GraphQL Toolbox is a low code web-based tool for building, testing, and exploring GraphQL APIs with the Neo4j GraphQL Library. Give it a try at graphql-toolbox.neo4j.io

Download Your Free Copy of Full Stack GraphQL Applications

Thanks to Neo4j for sponsoring my book it is available for free download at dev.neo4j.com/graphql-book. You can find the recording of the book club livestreams on the Neo4j YouTube channel.

Spatial Search Functionality With Neo4j

William Lyon — Wed, 04 Jan 2023 00:00:00 +0000

Radius Distance, Bounding Box, and Point in Polygon Search With Graphs

This post is based on my talk at NODES 2022, so you can watch the video recording instead:

Spatial Cypher Functions And The `Point` Type

Native geospatial functionality in Neo4j is based on support for a native Point type. The Point data type can be used to represent point data using either 3D or 2D geographic (WGS84) or cartesian coordinates reference systems (CRS). Points (or arrays of points) can be stored as properties on nodes or relationships in Neo4j. Cypher contains several functions for working with points, notably:

point() - for creating Point instances
point.distance() - calculates distance between two points. Can also be used for radius distance search when used in a WHERE predicate
point.withinBBox() - to find points within a given bounding box

Point instances can be created using the point() function in Cypher and passing an object that specifies latitude, longitude, and optionally height (when using geographic CRS) or x, y and optionally z (when using cartesian CRS). In this article we will focus on the 2D geographic CRS (using latitude an longitude).

Here we create a point instance by specifying latitude and longitude:

RETURN point(
  {latitude:49.38713, longitude:12.12711}
)

╒════════════════════════════════════════════════╕
│"point({latitude:49.38713, longitude:12.12711})"│
╞════════════════════════════════════════════════╡
│point({srid:4326, x:12.12711, y:49.38713})      │
└────────────────────────────────────────────────┘

To set a point property value we can use the SET Cypher command. Here we create a new PointOfInterest node representing a bakery and set the location property to a point value:

CREATE (p:PointOfInterest)
SET p.name = "West Side Bakery",
p.location = point(
  {latitude:49.38713, longitude:12.12711}
)
RETURN p

We can use the point.distance() function to search for our bakery. Here we search for nodes with the PointOfInterest label that are within 200 meters of a given latitude and longitude:

MATCH (p:PointOfInterest)
WHERE point.distance(
  p.location, point({latitude:49.38713, longitude:12.12711})
) < 200
RETURN p

Let's look at some real-world data to see how we can implement spatial search functionality in a simple web map application.

Spatial Search With Neo4j

A common feature for web applications that deal with geospatial data involves spatial search: finding things near other things or within the bounds of a box or perhaps within a polygon. Let's see how we can use Neo4j to search for points of interest on a map using Leaflet.js for the following types of spatial search:

Radius distance search
Within Bounding Box
Within Polygon

Code on GitHub: johnymontana/geospatial-graph-demos.

I wanted to use a real world dataset, so I pulled in points of interest from the Daylight Earth Table distribution of OpenStreetMap. The Daylight Earth Table is based on the Daylight OpenStreetMap distribution, but adds a data schema that classifies OpenStreetMap tags into a three level ontology: theme, class, and subclass. We'll be using the "poi" theme.

Currently there are almost 39 million points of interest in the dataset. The Daylight Earth Table is available in Parquet files stored in S3 so I used the pandas Python library to read the Parquet files from S3 into a DataFrame. By converting the DataFrame to Python dicts and passing in batches as Cypher parameters we can import the dataset in a few minutes.

# Here we define a function to split our DataFrame into batches for import

def insert_data(query, rows, batch_size=10000):
    total = 0
    batch = 0
    start = time.time()
    result = None

    while batch * batch_size < len(rows):
        res = conn.query(query, parameters = {'rows': rows[batch*batch_size:(batch+1)*batch_size].to_dict('records')}, db="osmpois")
        total += res[0]['total']
        batch += 1
        result = {"total": total, "batches": batch, "time": time.time()-start}
        print(result)
    return result

def insert_pois(rows, batch_size=10000):
    print(rows)
    query = '''
    UNWIND $rows AS row
    CREATE (p:PointOfInterest {name: row.names.local})
    CREATE (g:Geometry)
    SET g.location = point({latitude: toFloat(row.point[1]), longitude: toFloat(row.point[0]) })
    CREATE (g)<-[:HAS_GEOMETRY]-(p)
    SET g:Point
    CREATE (t:Tags)
    SET t += row.original_tags_dict
    CREATE (p)-[:HAS_TAGS]->(t)
    WITH *
    CALL apoc.create.addLabels(p, [row.class, row.subclass]) YIELD node
    RETURN COUNT(*) AS total
    '''
    return insert_data(query, rows, batch_size)

See the code on GitHub: johnymontana/daylight-earth-graph

Radius Distance Search

Now that we have all points of interest loaded into Neo4j we'll create a point index to enable fast index-backed spatial search operations.

CREATE POINT INDEX poiIndex
FOR (p:Point) ON (p.location)

Our first spatial search type is radius distance search: finding all points of interest within a given distance of a point.

Here we search for all points of interest within 200 meters of a point in Northern California.

PROFILE MATCH (p:Point)
WHERE
point.distance(p.location, point({latitude: 37.563434, longitude:-122.322255})) < 200
RETURN p{.*} AS point

By prepending PROFILE to our Cypher query we will render the database query plan as well as basic performance metrics for the query, such as memory usage, execution time, number of rows at each operation, and db hits. We can see that our point index is used resulting in an execution time of 1-2ms. Compare to the execution plan on the left which does not use an index and rather must scan every point of interest and takes about 4 seconds to execute. When we're dealing with millions of nodes it behooves us to make sure our spatial search operations are using an index for best performance.

Cypher Bounding Box Search

The next type of spatial search to consider is searching within a rectangular bounding box.

We will use the point.withinBBox Cypher predicate function to accomplish this. This function takes 3 arguments: a point value that defines a location and two other point values that define the bounding box's lower left and upper right points.

MATCH (p:Point)
WHERE point.withinBBox(
  p.location,
  point({longitude:-122.325447, latitude: 37.55948 }),  // Lower left point
  point({longitude:-122.314675 , latitude: 37.563596})) // Upper right point
RETURN p

We can prepend PROFILE to the above Cypher query to verify that we are indeed using the index for this spatial search operation.

Point In Polygon Search

Point in polygon search will return points within the bounds of an arbitrary polygon. Point in polygon search is not natively supported in Neo4j, however with some help on the client side of our application we can accomplish a polygon search while taking advantage of the point database index to ensure the search is fast. To accomplish this we will use the following steps:

Convert polygon to bounding box
Cypher withinBBox query
Filter results to those within polygon

Index backed point in polygon search can be accomplished by first converting the polygon to a bounding box, using Cypher's point.withinBBox predicate function to find points within the bounding box (using our database index), and then filtering the results on the client to the polygon bounds. Example code here, using Turf.js:

const polygon = layer.toGeoJSON();
var bbox = turf.bbox(polygon); // convert polygon to bounding box

// Within Bounding Box Cypher query
const cypher = `
    MATCH (p:Point)-[:HAS_GEOMETRY]-(poi:PointOfInterest)-[:HAS_TAGS]->(t:Tags)
    WHERE point.withinBBox(
        p.location,
        point({longitude: $lowerLeftLon, latitude: $lowerLeftLat }),
        point({longitude: $upperRightLon, latitude: $upperRightLat}))
    RETURN p { latitude: p.location.latitude,
               longitude: p.location.longitude,
               name: poi.name,
               categories: labels(poi),
               tags: t{.*}
            } AS point
          `;

var session = driver.session({
    database: "osmpois",
    defaultAccessMode: neo4j.session.READ,
});

session
    .run(cypher, {
        lowerLeftLat: bbox[1],
        lowerLeftLon: bbox[0],
        upperRightLat: bbox[3],
        upperRightLon: bbox[2],
    })
    .then((result) => {
        const bboxpois = [];
        result.records.forEach((record) => {
            const poi = record.get("point");
            var point = [poi.longitude, poi.latitude];
            bboxpois.push(point);
        });
        // filter results of bouding box query to polygon bounds
        const poisWithin = turf.pointsWithinPolygon(
            turf.points(bboxpois),
            polygon
        );

        poisWithin.features.forEach((e) => {
            L.marker([
                e.geometry.coordinates[1],
                e.geometry.coordinates[0],
            ])
            .addTo(map)
            .bindPopup("Polygon");
        })

You can find the code for the spatial search examples (including using the Leaflet.js web map with Neo4j) on GitHub: johnymontana/geospatial-graph-demos

Working With Line Geometries

So far we've been working with points, let's take a quick look at an example using line geometries. For this quick example I exported my Strava activities data and loaded into Neo4j.

To import data, first export user data from Strava then to add activities we will use the exported activities.csv file and load individual activity routes from the GPX files (first converted to geojson):

// Create Activity Nodes
LOAD CSV WITH HEADERS FROM "file:///activities.csv" AS row
MERGE (a:Activity {activity_id: row.`Activity ID`})
SET a.filename = row.Filename,
    a.activity_type = row.`Activity Type`,
    a.distance = toFloat(row.Distance),
    a.activity_name = row.`Activity Name`,
    a.activity_data = row.`Activity Date`,
    a.activity_description = row.`Activity Description`,
    a.max_grade = toFloat(row.`Max Grade`),
    a.elevation_high = toFloat(row.`Elevation High`),
    a.elevation_loss = toFloat(row.`Elevation Loss`),
    a.elevation_gain = toFloat(row.`Elevation Gain`),
    a.elevation_low = toFloat(row.`Elevation Low`),
    a.moving_time = toFloat(row.`Moving Time`),
    a.max_speed = toFloat(row.`Max Speed`),
    a.avg_grade = toFloat(row.`Average Grade`);

// Parse geojson geometries and create Geometry:Line nodes
MATCH (a:Activity)
WITH a WHERE a.filename IS NOT NULL AND a.filename CONTAINS ".gpx"
MERGE (n:Geometry {geom_id:a.activity_id })
MERGE (n)<-[:HAS_FEATURE]-(a)
WITH n,a
CALL apoc.load.json('file:///' + replace(a.filename, '.gpx', '.geojson')) YIELD value
UNWIND value.features[0].geometry.coordinates AS coord
WITH n, collect(point({latitude: coord[1], longitude: coord[0]})) AS coords
SET n.coordinates = coords
SET n:Line;

We store the routes as an array of point values to represent the line geometry of each route. Now, we can use the same spatial search techniques described above with our line geometries.

By using the list predicate any Cypher function in our query we can check if at least one point in the line are within the radius distance search distance.

WITH point({latitude: $latitude, longitude: $longitude}) AS radiusCenter
MATCH (g:Geometry)<-[:HAS_FEATURE]-(a:Activity)
WHERE any(
  p IN g.coordinates WHERE point.distance(p, radiusCenter) < $radius
)
RETURN *

Code for this example is available on GitHub here.

Routing With Graph Algorithms

One more use case I'd like to touch on besides spatial search is that of routing. We'll use the Graph Data Science Neo4j Sandbox to find airline routes between airports.

There are a number of relevant path finding algorithms we might consider. In this case we will use Dijkstra's algorithm, an extension of breadth first search that uses relationship weights to find the shortest path between two nodes, or airports in our case. Dijkstra's algorithm is one of many available in Neo4j's Graph Data Science library. First, we will project our airline route graph into an in-memory projected graph structure used by the Graph Data Science library:

CALL gds.graph.project(
  'routes-weighted',
  'Airport',
  'HAS_ROUTE',
  {
    relationshipProperties: 'distance'
  }
) YIELD graphName, nodeProjection, nodeCount,
  relationshipProjection, relationshipCount

Then, to search for the shortest path between two airports we use the gds.shortestPath.dijkstra procedure:

MATCH (source:Airport {iata: 'PUQ'}), (target:Airport {iata: 'HNL'})
CALL gds.shortestPath.dijkstra.stream('routes-weighted', {
  sourceNode: source,
  targetNode: target,
  relationshipWeightProperty: 'distance'
})
YIELD index, sourceNode, targetNode,
   totalCost, nodeIds, costs, path
RETURN
nodes(path) as path

Resources

Graph Data Visualization With GraphQL & react-force-graph

William Lyon — Wed, 29 Sep 2021 19:31:05 +0000

In this post we explore building an interactive graph data visualization using GraphQL as our data source, with the Lobsters GraphQL API we built previously. We'll be using Next.js, Vercel, and the react-force-graph library to add a graph visualization of users, tags, and articles posted to Lobste.rs. Previously we saw how to use GitHub Actions and Neo4j Aura to automate the import of Lobsters data into Neo4j, so check out the previous post if you're interested in that part.

We're building this application on the Neo4j livestream so you can check out the video recording to accompany this blog post here:

Force Directed Layouts & Graph Visualization

In the previous post we talked a bit about graph visualization with tools like Neo4j Bloom and Neo4j Browser, both of which are available in Neo4j Aura. Both Neo4j Bloom and Neo4j Browser (as well as many other tools and libraries) use a force-directed layout for graph visualization.

In a force-directed layout the nodes are positioned according to a physics simulation where connected nodes are attracted (think of the relationship acting as a spring) and nodes that are not connected to each other are pushed away from each other (like electrons). Force directed layouts are useful for graph visualization because they result in visual clusters of connected nodes that can help interpret the structure of the graph at a glance.

Graph Data Visualization With `react-force-graph`

The force-graph JavaScript library can be used to help build interactive data visualizations using a force-directed layout. It uses HTML5 Canvas for rendering and the d3-force layout algorithm. There are also 3D and AR/VR versions of the library, as well as a React flavor, which is what we'll be using.

Using `react-force-graph` with Next.js

First, let's install the react-force-graph-2d package:

yarn add react-force-graph-2d

We're using Next.js to build our application which will by default use server side rendering (SSR) to render our pages, however we don't want our graph visualization to be rendered on the server because it depends on elements of client JavaScript that runs in the browser to render the visualization. So we'll use the dynamic import feature of Next.js and disable SSR.

We'll create a simple module that imports and exports the ForceGraph2D module:

import ForceGraph2D from 'react-force-graph-2d';
export default ForceGraph2D;

Then import that module using a dynamic import:

import dynamic from 'next/dynamic';

const NoSSRForceGraph = dynamic(() => import('../lib/NoSSRForceGraph'), {
  ssr: false
});

The force-graph component expects our graph data in a certain format. Specifically, we'll need to provide the node and relationship data in an object that contains two arrays, nodes and links. The nodes array will contain objects that represent the nodes in the visualization and must include at least an id value that identifies uniqueness of the node. The links array represents our relationships connecting nodes and reference their source and target nodes using the node id.

Let's hardcode some fake node and link data to get going with a very basic visualization:

const myData = {
  nodes: [{ id: 'a' }, { id: 'b' }, { id: 'c' }],
  links: [
    { source: 'a', target: 'b' },
    { source: 'c', target: 'a' }
  ]
};

export default function Home() {
  return <NoSSRForceGraph graphData={myData} />;
}

Here's our basic graph visualization using react-force-graph and Next.js so far:

Now it's time to bring in our data!

Setting Up Apollo Client In Next.js

Since Next.js is a fullstack framework we'll be using GraphQL in two ways: 1) to implement our GraphQL server using the API Routes feature of Next.js, and 2) as a client of the GraphQL API to fetch and consume data via GraphQL.

We set up the GraphQL server in the previous episode so now it's time to set up the GraphQL client. We'll be using Apollo Client, which is one of the most popular GraphQL clients for React applications and will allow us to use some nice React Hooks for working with GraphQL data. First, let's install the @apollo/client package:

yarn add @apollo/client

Next, we need to create an Apollo Client instance and inject that client instance into the React component hierarchy using the ApolloProvider component so that the client instance will be available to all our React components. I covered setting up Apollo Client in a Next.js app in this blog post for the GRANDstack podcast application, so check that post out for a bit more detail, but we'll update pages/_app.js to create an Apollo Client instance:

import {
  ApolloProvider,
  ApolloClient,
  InMemoryCache,
  HttpLink
} from '@apollo/client';

const createApolloClient = () => {
  const link = new HttpLink({
    uri: '/api/graphql'
  });

  return new ApolloClient({
    link,
    cache: new InMemoryCache()
  });
};

function MyApp({ Component, pageProps }) {
  return (
    <ApolloProvider client={createApolloClient()}>
      <Component {...pageProps} />)
    </ApolloProvider>
  );
}

export default MyApp;

Pulling In GraphQL Data

Now we can use the Apollo Client React hooks in our app. Let's replace the placeholder data in our graph visualization with data populated from a GraphQL query of the Lobsters graph GraphQL API. We'll need to write a GraphQL query that will return the most recent articles (let's get the most recent 30), as well as the tags of the article and the user who submitted it.

{
    articles(options: { limit: 30, sort: { created: DESC } }) {
      __typename
      id
      url
      title
      created
      tags {
        __typename
        name
      }
      user {
        username
        avatar
        __typename
      }
    }
  }

Let's update index.js to use this GraphQL query to fetch data from our GraphQL endpoint using the useQuery hook. We'll also need to write a function (formatData) to convert the JSON data returned from our GraphQL query to the nodes and links format expected by the force-graph component. Finally, we'll save the result of that transformation into a React state variable (graphData) and pass that to the force-graph component. That way as we update the graphData state variable our visualization will update.

import dynamic from 'next/dynamic';
import { useQuery, gql } from '@apollo/client';
import { useState } from 'react';

const NoSSRForceGraph = dynamic(() => import('../lib/NoSSRForceGraph'), {
  ssr: false
});

const mostRecentQuery = gql`
  {
    articles(options: { limit: 30, sort: { created: DESC } }) {
      __typename
      id
      url
      title
      created
      tags {
        __typename
        name
      }
      user {
        username
        avatar
        __typename
      }
    }
  }
`;

const formatData = (data) => {
  // this could be generalized but let's leave that for another time

  const nodes = [];
  const links = [];

  if (!data.articles) {
    return;
  }

  data.articles.forEach((a) => {
    nodes.push({
      id: a.id,
      url: a.url,
      __typename: a.__typename,
      title: a.title
    });

    links.push({
      source: a.user.username,
      target: a.id
    });

    a.tags.forEach((t) => {
      nodes.push({
        id: t.name,
        __typename: t.__typename
      });
      links.push({
        source: a.id,
        target: t.name
      });
    });

    nodes.push({
      id: a.user.username,
      avatar: a.user.avatar,
      __typename: a.user.__typename
    });
  });

  return {
    // nodes may be duplicated so use lodash's uniqBy to filter out duplicates
    nodes: _.uniqBy(nodes, 'id'),
    links
  };
};

export default function Home() {
  const [graphData, setGraphData] = useState({ nodes: [], links: [] });

  const { data } = useQuery(mostRecentQuery, {
    onCompleted: (data) => setGraphData(formatData(data))
  });

  return (
    <NoSSRForceGraph
      graphData={graphData}
      nodeLabel={(node) => {
        return node.id;
      }}
      nodeAutoColorBy={'__typename'}
      nodeRelSize={8}
    />
  );
}

We also set the node label in the visualization to be the id value and use the nodeAutoColorBy configuration to assign colors by __typename (which map to node labels in the database). Here's what our visualization looks like now:

Adding More Nodes On Click

We want our visualization to be interactive and enable users to explore the graph. Let's add more data to the visualization when a user clicks on one of the tag nodes, allowing the user to find more articles for tags they might be interested in. To do this we'll write another GraphQL query (moreArticlesQuery) that will find the 10 most recent articles for a given tag. Then, we'll use the useLazyQuery hook from Apollo Client to execute this query when a user clicks on a tag node in the onNodeClick handler for the force-graph component. We'll also open the article in a new tab if the user clicks on an article node.

import dynamic from 'next/dynamic';
import { useQuery, useLazyQuery, gql } from '@apollo/client';
import { useState } from 'react';

const NoSSRForceGraph = dynamic(() => import('../lib/NoSSRForceGraph'), {
  ssr: false
});

const moreArticlesQuery = gql`
  query articlesByTag($tag: String) {
    articles(
      where: { tags: { name: $tag } }
      options: { limit: 10, sort: { created: DESC } }
    ) {
      __typename
      id
      url
      title
      created
      tags {
        __typename
        name
      }
      user {
        username
        avatar
        __typename
      }
    }
  }
`;

export default function Home() {
  const [graphData, setGraphData] = useState({ nodes: [], links: [] });
  const { data } = useQuery(mostRecentQuery, {
    onCompleted: (data) => setGraphData(formatData(data))
  });
  const [loadMoreArticles, { called, loading, data: newData }] = useLazyQuery(
    moreArticlesQuery,
    {
      onCompleted: (data) => {
        const newSubgraph = formatData(data);
        setGraphData({
          nodes: _.uniqBy([...graphData.nodes, ...newSubgraph.nodes], 'id'),
          links: [...graphData.links, ...newSubgraph.links]
        });
      }
    }
  );

  return (
    <NoSSRForceGraph
      graphData={graphData}
      nodeLabel={(node) => {
        return node.id;
      }}
      nodeAutoColorBy={'__typename'}
      nodeRelSize={8}
      onNodeClick={(node, event) => {
        console.log('You clicked me!');
        console.log(node);

        if (node.__typename === 'Tag') {
          console.log('Lode more articles');
          loadMoreArticles({ variables: { tag: node.id } });
        } else if (node.__typename == 'Article') {
          window.open(node.url, '_blank');
        }
      }}
    />
  );
}

Now here's what our visualization looks like after expanding a few tags.

It's a bit difficult to see at a glance what the tags and article titles are. Let's use text to represent the nodes in our visualization so we can see the relevant information without hovering over the nodes to see their labels.

Representing Nodes With Text

In order to change the visual representation of each node we'll need to override the nodeCanvasObject function. This function will allow us to the HTML Canvas API to style the nodes however we like. Using this example as a guide, let's use Canvas to style our nodes as text. We'll also use the user's avatar image to represent the user node.

return (
  <NoSSRForceGraph
    {/* other props omitted*/}
    nodeCanvasObject={(node, ctx, globalScale) => {
        if (node.__typename === "Tag" || node.__typename === "Article") {
          const label = node.title || node.id;
          const fontSize = 16 / globalScale;
          ctx.font = `${fontSize}px Sans-Serif`;
          const textWidth = ctx.measureText(label).width;
          const bckgDimensions = [textWidth, fontSize].map(
            (n) => n + fontSize * 0.2
          );
          ctx.fillStyle = `rgba(255, 255, 255, 0.8)`;
          ctx.fillRect(
            node.x - bckgDimensions[0] / 2,
            node.y - bckgDimensions[1] / 2,
            ...bckgDimensions
          );
          ctx.textAlign = "center";
          ctx.textBaseline = "middle";
          ctx.fillStyle = node.color;
          ctx.fillText(label, node.x, node.y);

          node.__bckgDimensions = bckgDimensions;
        } else if (node.__typename === "User") {
          const size = 12;
          const img = new Image();
          img.src = node.avatar;
          ctx.drawImage(img, node.x - size / 2, node.y - size / 2, size, size);
        }
    }}
  />
)

And now we can see at a glance the article titles and tags:

That's all for now, in the next post we'll take a look at improving some of the styling of our visualization and bring in some other visualization elements to help us explore the data and find articles we're interested in. You can find the current version of the app here and the GraphQL API is available here if you'd like to try some queries against it.

Resources

Building A GraphQL Server With Next.js API Routes

William Lyon — Tue, 27 Jul 2021 21:16:57 +0000

In the previous post we started working with data from the Lobsters social news aggregator site in Neo4j Aura with the ultimate goal of building a fullstack application that allows us to explore the data as an interactive graph visualization. In this post we continue our journey toward that fullstack goal, focusing on getting our API layer up and running using Next.js API routes, the Neo4j GraphQL library, Vercel and Neo4j Aura.

We're building this application on the Neo4j livestream so you can check out the video recording to accompany this blog post here:

Interactive Graph Visualization With Neo4j Bloom

Our goal is to build a web application that will demonstrate the power of data visualization when using social network data to, in this case, help us discover interesting and relevant content. Before we dive into building our fullstack application it's worth exploring what sort of graph data visualization tooling is available to us out of the box with Neo4j.

Perhaps the most relevant tool we might consider is Neo4j Bloom. Bloom is included in Neo4j Aura and Neo4j Desktop and allows the user to explore the graph visually without writing Cypher. Bloom is a standalone application that enables users to search for patterns, filter, explore and share graph visualizations.

While Bloom is a powerful tool for use with Neo4j, it's not quite what we want for this project as we want to build a most custom and bespoke experience. There are also other "graph apps" available in Neo4j Desktop, such as the Charts app and Neomap that we can use to build visualizations and dashboards. I covered these in a previous post: "Building An Election Night Dashboard With Neo4j Graph Apps: Bloom, Charts, & Neomap".

Next.js

Next.js is a fullstack React framework built and maintained by Vercel. Next.js includes many features out of the box that we typically need to set up in React applications - things like file-system routing, server side rendering, API routes, etc - which means we can focus on building our application and not boilerplate setup and configuration.

We covered Next.js in a previous blog post so I won't go over all the features now, but I'm a big fan of Next.js and use it with most of my new projects now.

`create-next-app`

The easiest way to get started with Next.js is to use the create-next-app CLI. This is a command line tool that enables us to quickly start building a new Next.js application. We can use it to create a new skeleton Next.js project or select from many of the example Next.js projects.

Let's use this to start a new Next.js application in our Lobsters Graph repository:

npx create-next-app next

We can now navigate to the next directory, and run yarn dev to start a local web server serving our Next.js application. We should see something like this with some placeholder content:

In this post we're going to focus on building the GraphQL API for our application, rather than the frontend so we won't cover anything React specific today. Instead, we'll be using Next.js' API Routes feature to build our GraphQL API.

Next.js API Routes

Next.js has support for creating API endpoints to add backend functionality to our Next.js application - it really is a fullstack framework after all. To create a new API route we just create a new file in pages/api that will be mapped to a new API endpoint.

The skeleton Next.js application we created with create-next-app includes an API route example in pages/api/hello.js:

// Next.js API route support: https://nextjs.org/docs/api-routes/introduction

export default function handler(req, res) {
  res.status(200).json({ name: 'John Doe' })
}

If we make a request to localhost:3000/api/hello we'll get back a simple JSON object:

{
    name: "John Doe"
}

Let's use this API route functionality to add a GraphQL endpoint to our Next.js application.

Creating A GraphQL Server In A Next.js API Route

Following the GraphQL example linked in the Next.js documentation, we'll use the micro package and apollo-server-micro to set up a simple GraphQL server as an API route.

First, we'll install the necessary dependencies:

yarn add apollo-server-micro micro graphql

Micro is an HTTP server that works well with Next.js and more importantly for our purposes has an Apollo Server implementation. To create a GraphQL server with Apollo Server we need to create two things: GraphQL type definitions that define the data available in the API, and GraphQL resolver functions that contain the logic for actually resolving GraphQL operations. Apollo Server takes these two inputs, combines them into an executable GraphQL schema and handles the HTTP network layer involved in serving a GraphQL API.

Let's create simple GraphQL type definitions and a single resolver function to get our API up and running as an API route:

import { gql, ApolloServer } from "apollo-server-micro";

const typeDefs = gql`
  type User {
    id: ID
  }

  type Query {
    getUser: User
  }
`;

const resolvers = {
  Query: {
    getUser: () => {
      return {
        id: "Foo",
      };
    },
  },
};

const apolloServer = new ApolloServer({
  typeDefs,
  resolvers,
});

const startServer = apolloServer.start();

export default async function handler(req, res) {

  await startServer;
  await apolloServer.createHandler({
    path: "/api/graphql",
  })(req, res);
}

export const config = {
  api: {
    bodyParser: false,
  },
};

We define a single Query field getUser and a single User type that has only one field, id and a resolver function that returns a single hardcoded User object.

Using GraphQL Playground With Apollo Server v3

In previous versions of Apollo Server by default the GraphQL Playground in-browser tool for exploring GraphQL APIs was available. However, GraphQL Playground has been deprecated for some time now and the latest release of Apollo Server, v3, instead links to the hosted Apollo Studio tool on the "landing page" of the GraphQL API (the page loaded when the GraphQL endpoint is loaded in a web browser).

Apollo Studio is great, but since we want this to be a public GraphQL API that anyone can explore I want GraphQL Playground to be served on the landing page. Fortunately, we can enable GraphQL Playground as a plugin with Apollo Server 3 with these changes:

import { ApolloServerPluginLandingPageGraphQLPlayground } from "apollo-server-core";

...

const apolloServer = new ApolloServer({
  typeDefs,
  resolvers,
  playground: true,
  plugins: [ApolloServerPluginLandingPageGraphQLPlayground()],
});

Now when we load localhost:3000/graphql/api in a web browser we should see the familiar GraphQL Playground tool. To verify that our GraphQL server is working properly we can run this query:

{
    getUser {
        id
    }
}

And we should see the simple result returned by our getUser resolver function:


{
    data: {
        getUser: {
            id: "Foo"
        }
    }
}

Now let's update our GraphQL schema from the placeholder to one that models our Lobsters graph data and works with Neo4j.

Using The Neo4j GraphQL Library

The Neo4j GraphQL library allows us to build Node.js GraphQL APIs backed by Neo4j without writing any resolvers. All we need to do is write GraphQL type definition that define the data model of our database and the Neo4j GraphQL library takes care of the rest - generating a full CRUD GraphQL API and resolvers and translating arbitrary GraphQL operations to database queries.

First, let's install a couple of additional dependencies, the Neo4j GraphQL library and the Neo4j JavaScript driver:

yarn add @neo4j/graphql neo4j-driver

Next, we'll need to create the GraphQL type definitions that map to the property graph model we're using for the Lobsters data. If we refer back to the previous post we can use the graph data model diagram we created using the Arrows.app tool:

The Neo4j GraphQL library uses the following conventions to map GraphQL type definitions to the property graph model:

GraphQL types map to node labels in the property graph model
GraphQL scalar fields map to node properties in the property graph model
GraphQL object and object array fields map to relationships in the property graph model
The @relationship directive is used in the GraphQL type definitions to encode the relationship type and direction in the property graph model

Applying these conventions we end up with the following GraphQL type definitions that map to our Lobsters property graph in Neo4j:

type User {
    username: String
    created: DateTime
    karma: Int
    about: String
    avatar: String
    articles: [Article] @relationship(type: "SUBMITTED", direction: OUT)
    invited: [User] @relationship(type: "INVITED_BY", direction: IN)
    invited_by: [User] @relationship(type: "INVITED_BY", direction: OUT)
  }

  type Article {
      id: ID
      url: String
      score: Int
      title: String
      comments: String
      created: DateTime
      user: User @relationship(type: "SUBMITTED", direction: IN)
      tags: [Tag] @relationship(type: "HAS_TAG", direction: OUT)
  }

  type Tag {
      name: String
      articles: [Article] @relationship(type: "HAS_TAG", direction: IN)
  }

Now we'll remove the resolver functions from our placeholder GraphQL schema since we don't need to write manual resolvers when using the Neo4j GraphQL library and replace our GraphQL type definitions with the ones we wrote above.

We'll also create a Neo4j JavaScript driver instance to connect to our Neo4j Aura database, using environment variables for the connection credentials and we'll pass our GraphQL type definitions to the Neo4jGraphQL class constructor to generate our GraphQL API.

We also make use of the @exclude directive in our GraphQL type definitions to prevent any mutations from being added to the schema - we want this to be a read-only API, at least for now.


import { gql, ApolloServer } from "apollo-server-micro";
import { ApolloServerPluginLandingPageGraphQLPlayground } from "apollo-server-core";
import {Neo4jGraphQL} from "@neo4j/graphql"
import neo4j from "neo4j-driver"
import 'ts-tiny-invariant' // importing this module as a workaround for issue described here: https://github.com/vercel/vercel/discussions/5846


const typeDefs = gql`
  type User @exclude(operations: [CREATE, UPDATE, DELETE]) {
    username: String
    created: DateTime
    karma: Int
    about: String
    avatar: String
    articles: [Article] @relationship(type: "SUBMITTED", direction: OUT)
    invited: [User] @relationship(type: "INVITED_BY", direction: IN)
    invited_by: [User] @relationship(type: "INVITED_BY", direction: OUT)
  }

  type Article @exclude(operations: [CREATE, UPDATE, DELETE]) {
      id: ID
      url: String
      score: Int
      title: String
      comments: String
      created: DateTime
      user: User @relationship(type: "SUBMITTED", direction: IN)
      tags: [Tag] @relationship(type: "HAS_TAG", direction: OUT)
  }

  type Tag @exclude(operations: [CREATE, UPDATE, DELETE]) {
      name: String
      articles: [Article] @relationship(type: "HAS_TAG", direction: IN)
  }
`;

const driver = neo4j.driver(
    process.env.NEO4J_URI,
    neo4j.auth.basic(process.env.NEO4J_USER, process.env.NEO4J_PASSWORD)
)

const neoSchema = new Neo4jGraphQL({typeDefs, driver})

const apolloServer = new ApolloServer({
  schema: neoSchema.schema,
  playground: true,
  introspection: true,
  plugins: [ApolloServerPluginLandingPageGraphQLPlayground()],
});

const startServer = apolloServer.start();

export default async function handler(req, res) {
  await startServer;
  await apolloServer.createHandler({
    path: "/api/graphql",
  })(req, res);
}

export const config = {
  api: {
    bodyParser: false,
  },
};

Next.js supports setting environment variables using .env files, so let's create a .env.local file where we'll add the credentials for our Neo4j Aura instance. We'll also set the DEBUG environment variable to enable debug logging for the Neo4j GraphQL library. This will log the generated Cypher queries among other things, which can be helpful to understand how the database queries are generated from GraphQL operations.

NEO4J_USER=neo4j
NEO4J_URI=neo4j+s://YOUR NEO4J AURA URI HERE
NEO4J_PASSWORD=YOUR NEO4J AURA PASSWORD HERE
DEBUG=@neo4j/graphql:*

We now have a GraphQL endpoint running locally at localhost:3000/api/graphql that we can use to fetch data from our Neo4j Aura database using GraphQL. Next, we'll deploy our Next.js application on Vercel so that it will be publicly accessible.

Deploying To Vercel

Vercel is a cloud platform that we'll use to build and deploy our Next.js application. The frontend React application (once we build it!) will be built and hosted on a CDN and our GraphQL API route will be automatically deployed as a serverless function.

Vercel integrates with GitHub so once we commit our changes we can add our Next.js application by selecting the GitHub repository in Vercel. Because we're using a bit of a monorepo setup and our Next.js application is not in the root directory of the repository we just need to tell Vercel that the root directory of our Next.js application is the next directory. We'll also add our Neo4j Aura connection credentials as environment variables, setting values for NEO4J_PASSWORD, NEO4J_URI, and NEO4J_USER in the Vercel project configuration.

Once we've added our project the Vercel build service will pull down our code from GitHub, build the project and deploy our Next.js application (static content to a CDN and our GraphQL API to a serverless function). Our project is automatically assigned a domain and SSL certificate! Because we connected our project via GitHub any commits and pull requests will trigger another build. Each build is assigned its own unique URL which means pull requests will be built and deployed as a "preview build" which we can test and share before deploying to our main domain. This is really great collaboration feature.

Since we added the GraphQL Playground plugin we can navigate to our Vercel project's URL in the browser https://lobste-rs-graph.vercel.app/api/graphql and test our GraphQL endpoint:

Now that we've got our GraphQL API up, running, and deployed on Vercel in the next post we'll start building out the frontend application, taking a look at graph data visualization in React with GraphQL. Be sure to subscribe to my newsletter to be notified when the next post is ready!

Resources

No Cost Data Scraping With GitHub Actions And Neo4j Aura

William Lyon — Wed, 21 Jul 2021 14:31:46 +0000

When working with data a common task is fetching data from some external source on a recurring basis and importing into a database for further analysis or as part of our application. Setting up servers to handle this can be time consuming and error prone. I recently came across a workflow using GitHub Actions and Neo4j Aura that makes this a breeze and with the free tiers of both GitHub Actions and Neo4j Aura is free to set up and run forever - great for side projects!

In this post we'll take a look at setting up this workflow to scrape data from the Lobsters news aggregator and import into a Neo4j Aura Free instance using GitHub Actions. We built this on the Neo4j livestream so check out the recording if you prefer video:

A Look At The Data - Lobsters

Lobsters is a social news aggregator. Users post links to articles and the community votes and comments on them in a discussion thread. An algorithm determines the ranking of the submissions, with the most recent and noteworthy articles floating to the front page. To be able to submit and comment users must be invited by an existing user and this user-invite graph is publicly available, helping to keep discussions civil and avoid fraudulent upvoting to game submission rankings.

I've been thinking about social networks recently and want to build an application to help explore relevant news articles using graph visualization so building something using data from Lobsters seems like a great fit. We want to import data about users and article submissions into Neo4j as the basis of our application, but how to get started? Fortunately, Lobsters makes two JSON endpoints available to fetch data about the newest and the "hottest" submissions. Each has a similar format and looks like this.

Now that we know what our JSON data looks like we can think about how to start importing this data into Neo4j Aura as a graph.

Neo4j Aura

Neo4j Aura is a managed cloud service from Neo4j allowing us to run Neo4j clusters in the cloud with a few clicks and handles things like backups, upgrades, and administration so we don't have to worry about it. Best of all for our purposes, Neo4j Aura has a free tier that will allow us to build this project without even putting in a credit card.

Let's sign-in to Neo4j Aura and create an Aura Free instance:

We'll immediately be given a generated password that we'll need to save to access our Neo4j Aura instance. We can change this later.

Next, our Aura instance will take a few moments to be provisioned.

Once our Neo4j Aura instance is ready we'll see the connection string in the dashboard. Let's open up Neo4j Browser to start working with our data.

Since we're working with JSON data we can make use of the apoc.load.json procedure from the APOC standard library to import a JSON file using Cypher. First, let's just pull this file in to make sure we can parse it. This Cypher statement will parse the newest Lobsters submissions and return an array of objects that we can work with in Cypher:

CALL apoc.load.json("https://lobste.rs/newest.json") YIELD value
RETURN value

If we run this in Neo4j Browser we'll see the parsed array of objects returned. We haven't actually created any data in the database yet - we're just parsing the JSON file and returning the results.

We'll use Cypher to define the graph structure we want to create from this data, but we first need to think a bit about how we want to model this data as a graph. I think of graph modeling as a multi-step process, at the highest level the steps are:

Identify the entities in our data - these become nodes.
Identify how these entities are connected - these connections become relationships.
Identify the pieces of data that describe the entities or connections - these become properties.

Arrows.app is a great tool for drawing and storing these graph models. Here's what our graph model for the Lobsters data looks like in Arrows:

Now we can write the Cypher to import our data according to this model. We'll use apoc.load.json to parse the JSON file, then use UNWIND to iterate over this array of objects. We'll then use the MERGE Cypher clause to add users, articles, and tags to the database. The MERGE statement allows us to avoid creating duplicates in the graph - with MERGE only patterns that don't already exist in the graph are created.

This makes our import statement idempotent - we can run it over and over again on the same data and we won't make any changes to the database unless the data changes.

CALL apoc.load.json("https://lobste.rs/newest.json") YIELD value
UNWIND value AS article
MERGE (s:User {username: article.submitter_user.username})
ON CREATE SET s.about = article.submitter_user.about,
              s.created = DateTime(article.submitter_user.created_at),
              s.karma = article.submitter_user.karma,
              s.avatar_url = "https://lobsete.rs" + article.submitter_user.avatar_url
MERGE (i:User {username: article.submitter_user.invited_by_user})
MERGE (i)<-[:INVITED_BY]-(s)
MERGE (a:Article {short_id: article.short_id})
SET a.url = article.url,
    a.score = article.score,
    a.created = DateTime(article.created_at),
    a.title = article.title,
    a.comments = article.comments_url
MERGE (s)-[:SUBMITTED]->(a)
WITH article, a
UNWIND article.tags AS tag
MERGE (t:Tag {name: tag})
MERGE (a)-[:HAS_TAG]->(t)

To verify we've imported this data correctly we can visualize it in Neo4j Browser - let's check to make sure the property values are what we expect and makes sense.

That's great, but the Lobsters data will be continually changing as new articles are submitted and voted on - we want to import it on an ongoing basis. Let's use GitHub Actions to do this!

GitHub Actions And Flat Graph

GitHub Actions allows us to define workflows that are triggered by GitHub events (like commits, pull requests, etc). We can write custom code to run when an Action is triggered or choose from existing Actions published by community members in the GitHub Action Marketplace. While commonly used for CI/CD workflows, we can also schedule Actions to run at a specified interval.

The GitHub team recently released the Flat Data project which aims to simplify data and ETL workflows. Flat Data includes the Flat Data GitHub Action, a VSCode extension for creating Flat Data workflows, and a web app for viewing data. The Flat Data Action allows us to schedule a GitHub action to periodically fetch data via a URL or SQL statement, check the data into git, and run a postprocessing step to, for example, insert data into a database.

This sounds perfect for what we want to accomplish with Lobsters and Neo4j Aura - let's give it a try! First, we'll sign into GitHub and create a new repository:

Because we selected "Add a README file" we can immediately start to make commits to our repo and use the built-in editor in the GitHub web UI to set up our Flat Data Action. We can also clone the repo locally, but I'll just use the built-in text editor.

To create a new GitHub action we'll add a new file .github/workflows/lobsters.yml. In this YAML file we will

Give a name to our new Action - let's call it "Lobsters Data Import".
Identity the events we want to trigger this action. We want it to run anytime this YAML file is updated and also schedule it to run every 60 minutes.
We then define the steps we want to run in our Action:

First, we'll check out the repo using the actions/checkout@v2 action
Then we'll use the githubocto/flat@v2 Action to fetch our Lobsters JSON file and check it into our repo as newest.json

name: Lobsters Data Import

on:
  push:
    paths:
      - .github/workflows/lobsters.yml
  workflow_dispatch:
  schedule:
    - cron: '*/60 * * * *'

jobs:
  scheduled:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Fetch newest
        uses: githubocto/flat@v2
        with:
          http_url: https://lobste.rs/newest.json
          downloaded_filename: newest.json

Using the in-browser editor in GitHub we should see something like this:

Our GitHub Action will run immediately after committing the new file and in a few seconds we'll see a new commit, checking in the newest.json file with the data from Lobsters.

Great - we've fetched the newest Lobsters article submissions and checked them into our repository and scheduled a GitHub Action to refresh this data every hour. Now we're ready to update our GitHub Action to import data into Neo4j.

First, we'll need to add secrets to our GitHub repository so our Action can connect to our Neo4j Aura instance and we don't have to expose our connection credentials. In GitHub select the "Settings" tab then navigate to the "Secrets" section and select the "New repository secret" button.

We'll create three secrets NEO4J_USER, NEO4J_PASSWORD, and NEO4J_URI with the connection credentials specific to the Neo4j Aura instance we created earlier.

We'll now be able to reference these secrets values in the YAML file where we define our GitHub Action to connect to our Neo4j Aura instance.

The Flat Graph GitHub Action

The Flat Data GitHub Action includes support for a post-processing step that allows us to run a JavaScript or Python script after the data is fetched. We could write a simple script to use the Neo4j language driver to connect to our Neo4j Aura instance and run a Cypher import statement, passing in the JSON data as a Cypher parameter, however there is also the Flat Graph GitHub Action which does just this.

Flat Graph is designed to work with the Flat Data GitHub Action and allows us to declare the Cypher import statement we want to run and our Neo4j Aura connection credentials as another step in our GitHub Action. Flat Graph will load the specified JSON file and pass it as a Cypher parameter called $value so our Cypher import statement just needs to reference this Cypher parameter to work with the data fetched by Flat Data in the previous step of our Action.

We'll use the Cypher statement we wrote above using apoc.load.json, but adapt it to use this convention. We'll also reference the Neo4j Aura connection credentials we defined as GitHub secrets. Let's update our lobsters.yml file:

name: Lobsters Data Import

on:
  push:
    paths:
      - .github/workflows/lobsters.yml
  workflow_dispatch:
  schedule:
    - cron: '*/60 * * * *'

jobs:
  scheduled:
    runs-on: ubuntu-latest
    steps:
      - name: Check out repo
        uses: actions/checkout@v2
      - name: Fetch newest
        uses: githubocto/flat@v2
        with:
          http_url: https://lobste.rs/newest.json
          downloaded_filename: newest.json
      - name: Neo4j import
        uses: johnymontana/flat-graph@v1.2
        with:
          neo4j-user: ${{secrets.NEO4J_USER}}
          neo4j-password: ${{secrets.NEO4J_PASSWORD}}
          neo4j-uri: ${{secrets.NEO4J_URI}}
          filename: newest.json
          cypher-query: >
            UNWIND $value AS article
            MERGE (s:User {username: article.submitter_user.username})
            ON CREATE SET s.about = article.submitter_user.about,
              s.created = DateTime(article.submitter_user.created_at),
              s.karma = article.submitter_user.karma,
              s.avatar_url = "https://lobsete.rs" + article.submitter_user.avatar_url
            MERGE (i:User {username: article.submitter_user.invited_by_user})
            MERGE (i)<-[:INVITED_BY]-(s)
            MERGE (a:Article {short_id: article.short_id})
            SET a.url = article.url,
                a.score = article.score,
                a.created = DateTime(article.created_at),
                a.title = article.title,
                a.comments = article.comments_url
            MERGE (s)-[:SUBMITTED]->(a)
            WITH article, a
            UNWIND article.tags AS tag
            MERGE (t:Tag {name: tag})
            MERGE (a)-[:HAS_TAG]->(t)

Once we commit the change our Action will run - loading the latest Lobsters article data into our Neo4j Aura instance. This Action will run each hour updating our database with the latest Lobsters data.

What's Next?

We've now set up a serverless data scraping workflow to fetch all new articles submitted to Lobsters and import into Neo4j Aura using GitHub Actions. Next, we'll take a look at data visualization options and we begin to explore this dataset and build a web application to help us find relevant information. If you'd like to follow along we'll be coding this on the Neo4j Livestream on Twitch and YouTube. You can also subscribe to my newsletter to be notified as new posts are up. Happy graphing!

Resources

johnymontana / lobste.rs-graph

Importing Lobste.rs data into Neo4j Aura using GitHub Actions

Getting Started With Next.js and GraphQL Authentication | Building GRANDcast.FM Episode 4

William Lyon — Sun, 24 Jan 2021 21:41:15 +0000

This is the fourth post in a series about building a podcast application using GRANDstack. Check out the previous episodes here:

Episode 1: "Podcast Search GraphQL API With Neo4j And The Podcast Index"

Episode 2: "GRANDcast.FM: User Auth & Podcast Subscribe Functionality"

Episode 3: "Parsing And Importing XML With Neo4j: Adding Episodes and Playlists"

In this post we turn our attention to the frontend of the podcast application we've been building on the Neo4j livestream. So far we’ve focused on the backend and GraphQL API layer. This week we got started on the front end, using the Next.js React framework. You can find the livestream recording embedded below or on the Neo4j Youtube channel.

In this post we'll focus on getting started with Next.js, fetching data using GraphQL in our Next.js application, and setting up client-side authentication with GraphQL in Next.js so we can sign in and make authenticated requests to our GraphQL API.

What Is Next.js?

Next.js is a web framework built on top of React that adds many features and conventions that aren't available by default with React. Things like server side rendering, file based routing, image optimization, code splitting and bundling, and adding server-side API routes are things we eventually need to deal with as our React application becomes more complex. With Next.js we have all these things (and lots more) available out of the box. I like to think of Next.js as React with batteries included.

Getting Started With Next.js

The easiest way to get started with Next.js is using the create-next-app CLI. Similar to create-react-app or create-grandstack-app, we can create an initial Next.js application from the command line.

npx create-next-app next-app

The create-next-app command can take an example argument to specify an example template to use. There are two example Next.js templates for using Next.js with Neo4j:

We're going to skip these Neo4j specific templates since we've already built out our GraphQL API. In a future post we'll see how we can move our GraphQL API into Next.js, taking advantage of the serverless function deployment built into Next.js API routes.

To start our Next.js application we'll run

npm run dev

which will start a local server at localhost:3000 serving our Next.js application.

GraphQL Data Fetching With Next.js

The first thing we want to do is make data fetching queries with GraphQL and display podcast data in our Next.js application. To do this we'll use Apollo Client. First, let's install Apollo Client, by default this will include the React hooks integration for Apollo Client as well.

npm i @apollo/client

In React applications we make use of the Provider pattern and the React Context API to make data available throughout the React component hierarchy. The React integration for Apollo Client includes an ApolloProvider component that we can use to inject an Apollo Client instance into the React component hierarchy, which will make Apollo Client available to any component in our application. To do this we need to create an Apollo Client instance and wrap our application's root component in the ApolloProvider component. In Next.js we can do this in the _app.js file.

import '../styles/globals.css'
import {
  ApolloProvider,
  ApolloClient,
  InMemoryCache,
  HttpLink,
} from '@apollo/client'

function createApolloClient() {
  const link = new HttpLink({
    uri: 'http://localhost:4001/graphql',
  })

  return new ApolloClient({
    link,
    cache: new InMemoryCache(),
  })
}

function MyApp({ Component, pageProps }) {
  return (
    <ApolloProvider client={createApolloClient()}>
      <Component {...pageProps} />
    </ApolloProvider>
  )
}

export default MyApp

Note the use of an HttpLink instance. With Apollo Client, links are composable units, similar to middleware, that allow us to inject logic into the networking layer of a GraphQL request. Here we create a link to point Apollo Client to our GraphQL API running at localhost:4001/graphql. Later, we'll use the link to add an authorization header that includes an auth token once a user has signed in to the application, but for now we'll start with making unauthenticated GraphQL requests.

Making GraphQL Queries With Apollo Client In A Next.js Page

With Next.js' file based routing system we can create new pages just by creating a file in the pages directory and exporting a React component. Let's create a podcasts page to display a list of available podcasts. We'll make use of the useQuery Apollo Client hook to execute a GraphQL query to find all podcasts in the database and return their title.

import { useQuery, gql } from '@apollo/client'

const PodcastQuery = gql`
  {
    Podcast {
      title
    }
  }
`

const Podcasts = () => {
  const { data } = useQuery(PodcastQuery)

  return (
    <div>
      <ul>
        {data?.Podcast.map((v) => {
          return <li key={v.title}>{v.title}</li>
        })}
      </ul>
    </div>
  )
}

export default Podcasts

Next.js will automatically add a route at /podcasts to render our new page.

There are only a few podcasts in the database since they are only added to the database when a user subscribes. Refer to Episode 2 to see how we implemented podcast subscribe functionality.

Slight Detour - Setting Up The GraphQL VS Code Extension

If you use the VS Code editor it can be helpful to make use of the GraphQL VS Code extension. This extension will enable syntax highlighting in .graphql files, gql template tags, and can also be used to execute GraphQL queries that are tagged in your JavaScript files. To use the GraphQL VS Code extension you'll need to install it from the extension marketplace here. Then you'll also need to configure your project's GraphQL APIs in a GraphQL config file. Here's a simple example for this project:

schema: 'api/src/schema.graphql'
extensions:
  endpoints:
    default:
      url: http://localhost:4001/graphql

This will now allow us to execute GraphQL queries from VS Code that we've tagged with the gql template tag:

Authenticated GraphQL Requests In Next.js

We've seen how to make use of Apollo Client in our Next.js application to render a list of available podcasts, however this only works for unauthenticated GraphQL requests. We need to allow users to sign in to our application and then once they've signed in expose the user-specific functionality that we've built into our GraphQL API layer: podcast subscribe, new episode feeds, and create/view/update episode playlists.

To handle authentication and the associated state (Is the user signed in? What is the user's auth token?) we will make use of the React Context API and the Provider pattern again, this time creating our own AuthProvider that will allow us to keep track of and update authentication related state. Specifically, we'll need to:

Execute the login GraphQL mutation operation to sign a user into the application.
The login GraphQL mutation will return a JWT token when a user successfully signs in. We'll need to keep track of that token.
When a user is signed in and a valid JWT is generated we need to add the token to all GraphQL requests as an authentication header.
Allow a user to sign out.

All of this functionality will then be made available to any of our application components by importing the useAuth hook that we will create in our authentication provider.

import React, { useState, useContext, createContext } from 'react'
import {
  ApolloProvider,
  ApolloClient,
  InMemoryCache,
  HttpLink,
  gql,
} from '@apollo/client'

const authContext = createContext()

export function AuthProvider({ children }) {
  const auth = useProvideAuth()

  return (
    <authContext.Provider value={auth}>
      <ApolloProvider client={auth.createApolloClient()}>
        {children}
      </ApolloProvider>
    </authContext.Provider>
  )
}

export const useAuth = () => {
  return useContext(authContext)
}

function useProvideAuth() {
  const [authToken, setAuthToken] = useState(null)

  const isSignedIn = () => {
    if (authToken) {
      return true
    } else {
      return false
    }
  }

  const getAuthHeaders = () => {
    if (!authToken) return null

    return {
      authorization: `Bearer ${authToken}`,
    }
  }

  const createApolloClient = () => {
    const link = new HttpLink({
      uri: 'http://localhost:4001/graphql',
      headers: getAuthHeaders(),
    })

    return new ApolloClient({
      link,
      cache: new InMemoryCache(),
    })
  }

  const signIn = async ({ username, password }) => {
    const client = createApolloClient()
    const LoginMutation = gql`
      mutation signin($username: String!, $password: String!) {
        login(username: $username, password: $password) {
          token
        }
      }
    `

    const result = await client.mutate({
      mutation: LoginMutation,
      variables: { username, password },
    })

    console.log(result)

    if (result?.data?.login?.token) {
      setAuthToken(result.data.login.token)
    }
  }

  const signOut = () => {
    setAuthToken(null)
  }

  return {
    setAuthToken,
    isSignedIn,
    signIn,
    signOut,
    createApolloClient,
  }
}

Note how we make use of the HttpLink to add the authentication header to GraphQL requests when a user is signed in. We also wrap the ApolloProvider component in our AuthProvider which will make Apollo Client and the authentication functionality available to any child components in the React component hierarchy. We'll need to update _app.js to use this new AuthProvider:

import '../styles/globals.css'
import { AuthProvider } from '../lib/auth.js'

function MyApp({ Component, pageProps }) {
  return (
    <AuthProvider>
      <Component {...pageProps} />
    </AuthProvider>
  )
}

export default MyApp

The Sign In Flow

Now that we've implemented authentication functionality in our AuthProvider we can implement the sign in flow in our index page. First, we'll want to import the new useAuth hook we created:

import { useAuth } from '../lib/auth.js'

Now, let's add a SignIn component with a form for the user to submit their username and password. When the user submits the form we'll call the signIn function made available in auth.js which will execute the login GraphQL mutation and generate an auth JSON Web Token (JWT).

const SignIn = () => {
  const [username, setUsername] = useState('')
  const [password, setPassword] = useState('')

  const { signIn, signOut } = useAuth()

  function onSubmit(e) {
    e.preventDefault()
    signIn({ username, password })
  }

  return (
    <div>
      <form onSubmit={onSubmit}>
        <input
          type="text"
          placeholder="username"
          onChange={(e) => setUsername(e.target.value)}
        ></input>
        <input
          type="password"
          placeholder="password"
          onChange={(e) => setPassword(e.target.value)}
        ></input>
        <button type="submit">Sign In</button>
      </form>
    </div>
  )
}

Once a user is signed in we want to show their podcast episode feed - the most recent podcast episodes across all the podcasts they subscribe to. This is available via the episodeFeed GraphQL query field. We'll create a FeedQuery component to execute this query and render the results in a list. Since the user is signed in, the appropriate auth token will be added in the header of the GraphQL request, making this an authenticated GraphQL request against our API.

const FeedQuery = gql`
  {
    episodeFeed(first: 50) {
      id
      title
      audio
      podcast {
        title
      }
    }
  }
`

const EpisodeFeed = () => {
  const { data } = useQuery(FeedQuery)
  const { signOut } = useAuth()
  return (
    <div>
      <h1>Episode Feed</h1>
      <ul>
        {data?.episodeFeed.map((v) => {
          return <li key={v.id}>{v.title}</li>
        })}
      </ul>
      <button onClick={() => signOut()}>Sign Out</button>
    </div>
  )
}

Next, we'll make use of the isSignedIn function made available via useAuth to determine whether to render the sign in form or the episode feed component.

export default function Home() {
  const { isSignedIn } = useAuth()
  return (
    <div className={styles.container}>
      <Head>
        <title>GRANDcast.FM</title>
        <link rel="icon" href="/favicon.ico" />
      </Head>

      <main className={styles.main}>
        <h1>GRANDcast.FM</h1>
        {!isSignedIn() && <SignIn />}
        {isSignedIn() && <EpisodeFeed />}
      </main>
    </div>
  )
}

Now, when we first load the page we're presented with a sign in form that will execute the login GraphQL mutation when submitted. If a successful login, then the GraphQL server will generate an authorization JWT and return the token to the client. Our auth provider will store that token and attach it in the authorization header of all future GraphQL requests, until we log out.

Once we're authenticated, instead of the login form, we're presented with a list of the most recent podcast episodes for all podcasts to which we subscribe. We can click the "Sign Out" button, which will update the authentication state in our application, remove the token from future GraphQL requests, and we'll be presented with the sign in form again.

Now that we've set up client-side authentication in our Next.js application we can start to explore the different data-fetching patterns available to us in Next.js. We skipped over this functionality in this post but the different server data-fetching models in Next.js are some of the most interesting features of the framework. We'll dig into this in the next episode.

Be sure to subscribe to the Neo4j livestream on Twitch or Youtube to keep up to date as we build out more features in GRANDcast.FM. What else would you like us to explore on the livestream? Let us know on Twitter.

Resources

Parsing And Importing XML With Neo4j: Adding Episodes and Playlists To GRANDcast.FM Podcast App

William Lyon — Sun, 10 Jan 2021 04:29:27 +0000

This is the third post in a series about building a podcast application using GRANDstack. Check out the first post, "Podcast Search GraphQL API With Neo4j And The Podcast Index" where we start building the GraphQL API and implement podcast search functionality and the second post, "GRANDcast.FM: User Auth & Podcast Subscribe Functionality"

In this post, we continue building out functionality in the GraphQL API powering our podcast application. This week we focus on adding episodes by parsing podcast XML feeds and adding user playlists to the GraphQL API. We added this functionality during the Neo4j livestream. You can watch the recording of that here:

Working With Podcast Feeds

A podcast feed is an XML document that conforms to the RSS 2.0 specification. The basic structure of an RSS document consists of a top-level <rss> element, followed by a <channel> element which contains metadata about the podcast (title, description, link, etc). We already have this data for each podcast that a user has subscribed to, fetched from the Podcast Index API and added to the database. What we're interested in is the next sub-element of the document, <items>, which consists of one or more <item> elements.

Each <item> element represents a podcast episode and contains the following elements (technically these are all optional according to the specification, and other elements may be included as well):

<title> - The title of the podcast episode
<link> - The URL of the podcast episode (not to be confused with the audio file URL)
<description> - A synopsis of the episode
<enclosure> - A media object (such as an audio file) attached to the episode
<guid> - A globally unique string used to identify the episode
<pubDate> - Indicates when the episode was published, using RFC822 datetime format.

Now that we know the structure of a podcast feed XML document, let's start parsing them and adding episodes to the database.

Parsing XML With Neo4j

We can parse and import XML data directly into Neo4j using the apoc.load.xml procedure. This procedure takes the URL for the XML file and returns a single nested map representing the XML document using _type, _text, and _children keys to represent values, metadata, and element type as represented in XML. The APOC documentation has a simple example showing how this works. Here we use Cypher to filter for the elements of each episode that we want to capture, using the Graphistania podcast as an example:

MATCH (p:Podcast)
CALL apoc.load.xml("http://feeds.soundcloud.com/users/soundcloud:users:141739624/sounds.rss")
YIELD value
WITH [x in value._children[0]._children WHERE x._type = "item"] AS episodes
UNWIND episodes AS episode
WITH
    [X in episode._children WHERE X._type = "title"][0]._text AS title,
    [X in episode._children WHERE X._type = "summary"][0]._text AS summary,
    [X in episode._children WHERE X._type = "link"][0]._text AS link,
    [X in episode._children WHERE X._type = "image"][0].href AS image,
    [X in episode._children WHERE X._type = "enclosure" AND X.type CONTAINS "audio"][0].url AS audio,
    [X in episode._children WHERE X._type = "pubDate"][0]._text AS pubDate,
    [X in episode._children WHERE X._type ="guid"][0]._text AS guid
RETURN *

╒══════════════════════════════════════════════════════════════════════╤══════════════════════════════════════╤══════════════════════════════════════════════════════════════════╤══════════════════════════════════════════════════════════════════════╤═════════════════════════════════╤══════════════════════════════════════════════════════════════════════╤══════════════════════════════════════════════════════════════════════╕
│"audio"                                                               │"guid"                                │"image"                                                           │"link"                                                                │"pubDate"                        │"summary"                                                             │"title"                                                               │
╞══════════════════════════════════════════════════════════════════════╪══════════════════════════════════════╪══════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════╪═════════════════════════════════╪══════════════════════════════════════════════════════════════════════╪══════════════════════════════════════════════════════════════════════╡
│"http://www.podtrac.com/pts/redirect.mp3/feeds.soundcloud.com/stream/9│"tag:soundcloud,2010:tracks/935068060"│"http://i1.sndcdn.com/avatars-000135096101-qekfg1-original.png"   │"https://soundcloud.com/graphistania/graphistania-20-episode-11"      │"Tue, 24 Nov 2020 00:00:00 +0000"│"Graphistania 2.0 - Episode 11 by The Neo4j Graph Database Community" │"Graphistania 2.0 - Episode 11"                                       │
│35068060-graphistania-graphistania-20-episode-11.mp3"                 │                                      │                                                                  │                                                                      │                                 │                                                                      │                                                                      │
├──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│"http://www.podtrac.com/pts/redirect.mp3/feeds.soundcloud.com/stream/9│"tag:soundcloud,2010:tracks/926771035"│"http://i1.sndcdn.com/avatars-000135096101-qekfg1-original.png"   │"https://soundcloud.com/graphistania/graphistania-20-episode-10"      │"Thu, 12 Nov 2020 08:08:48 +0000"│"Graphistania 2.0 - Episode 10 by The Neo4j Graph Database Community" │"Graphistania 2.0 - Episode 10"                                       │
│26771035-graphistania-graphistania-20-episode-10.mp3"                 │                                      │                                                                  │                                                                      │                                 │                                                                      │                                                                      │
├──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│"http://www.podtrac.com/pts/redirect.mp3/feeds.soundcloud.com/stream/9│"tag:soundcloud,2010:tracks/905615800"│"http://i1.sndcdn.com/avatars-000135096101-qekfg1-original.png"   │"https://soundcloud.com/graphistania/graphistania-20-episode-9"       │"Tue, 06 Oct 2020 00:00:00 +0000"│"Graphistania 2.0 - Episode 9 by The Neo4j Graph Database Community"  │"Graphistania 2.0 - Episode 9"                                        │
│05615800-graphistania-graphistania-20-episode-9.mp3"                  │                                      │                                                                  │                                                                      │                                 │                                                                      │                                                                      │
├──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│"http://www.podtrac.com/pts/redirect.mp3/feeds.soundcloud.com/stream/8│"tag:soundcloud,2010:tracks/889154860"│"http://i1.sndcdn.com/avatars-000135096101-qekfg1-original.png"   │"https://soundcloud.com/graphistania/graphistania-20-episode-8"       │"Tue, 08 Sep 2020 00:00:00 +0000"│"Graphistania 2.0 - Episode 8 by The Neo4j Graph Database Community"  │"Graphistania 2.0 - Episode 8"                                        │
│89154860-graphistania-graphistania-20-episode-8.mp3"                  │                                      │                                                                  │                                                                      │                                 │                                                                      │                                                                      │
├──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│"http://www.podtrac.com/pts/redirect.mp3/feeds.soundcloud.com/stream/8│"tag:soundcloud,2010:tracks/853044658"│"http://i1.sndcdn.com/avatars-000135096101-qekfg1-original.png"   │"https://soundcloud.com/graphistania/graphistania-20-episode-7"       │"Fri, 10 Jul 2020 00:00:00 +0000"│"Graphistania 2.0 - episode 7 by The Neo4j Graph Database Community"  │"Graphistania 2.0 - episode 7"                                        │
│53044658-graphistania-graphistania-20-episode-7.mp3"                  │                                      │                                                                  │                                                                      │                                 │                                                                      │                                                                      │
├──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤
│"http://www.podtrac.com/pts/redirect.mp3/feeds.soundcloud.com/stream/7│"tag:soundcloud,2010:tracks/791209915"│"http://i1.sndcdn.com/avatars-000135096101-qekfg1-original.png"   │"https://soundcloud.com/graphistania/graphistania-20-episode-6"       │"Mon, 06 Apr 2020 00:00:00 +0000"│"Graphistania 2.0 - episode 6 by The Neo4j Graph Database Community"  │"Graphistania 2.0 - episode 6"                                        │
│91209915-graphistania-graphistania-20-episode-6.mp3"                  │                                      │                                                                  │                                                                      │                                 │                                                                      │                                                                      │
├──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────┼──────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼─────────────────────────────────┼──────────────────────────────────────────────────────────────────────┼──────────────────────────────────────────────────────────────────────┤

Now that we can parse each podcast feed we're ready to add these episodes to the database.

Adding Podcast Episodes To The Graph

We'll use the apoc.load.xml procedure to parse the podcast feeds for any podcasts currently stored in the database and create Episode nodes in the graph with the relevant details for each episode stored as node properties. We'll connect the episode to its podcast using a HAS_EPISODE relationship.

Here we create an Episode node for each podcast episode parsed from the feed. Note how we make use of the apoc.date.parse function when storing the pubDate property as a datetime type.

MATCH (p:Podcast)
call apoc.load.xml(p.feedURL)
YIELD value
UNWIND value._children AS foo
WITH p, [x in foo._children WHERE x._type = "item"] AS episodes
UNWIND episodes AS episode
WITH p,
       [X in episode._children WHERE X._type = "title"][0]._text AS title,
       [X in episode._children WHERE X._type = "summary"][0]._text AS summary,
       [X in episode._children WHERE X._type = "link"][0]._text AS link,
       [X in episode._children WHERE X._type = "image"][0].href AS image,
       [X in episode._children WHERE X._type = "enclosure" AND X.type CONTAINS "audio"][0].url AS audio,
       [X in episode._children WHERE X._type = "pubDate"][0]._text AS pubDate,
       [X in episode._children WHERE X._type ="guid"][0]._text AS guid

MERGE (e:Episode {id: guid})
SET e.title = title,
    e.summary = summary,
    e.link = link,
    e.image = image,
    e.audio = audio,
    e.pubDate = datetime({epochMillis: apoc.date.parse(pubDate, "ms", "EEE, dd MMM yyyy HH:mm:ss z")})
MERGE (e)<-[:HAS_EPISODE]-(p)

Adding Episodes When A User Subscribes To A Podcast

Now that we have the Cypher statement to parse podcast feeds and insert episodes into the database we need to update our GraphQL API so that when a podcast is added to the database we start parsing the feed. There are a few different ways we could approach this, perhaps creating a database trigger or a background job to parse the feed. Instead of those options let's update the subscribeToPodcast GraphQL mutation to include the Cypher for parsing the podcast feed and adding episodes to the graph when a user subscribes to the podcast. To do that we'll add the above Cypher statement to the subscribeToPodcast mutation @cypher statement:

type Mutation {
  signup(username: String!, password: String!): AuthToken
  login(username: String!, password: String!): AuthToken
  subscribeToPodcast(itunesId: String!): Podcast
  @cypher(
    statement: """
    // First we fetch the podcast details from the podcast index
    WITH toString(timestamp()/1000) AS timestamp
    WITH {
    `User-Agent`: 'GRANDstackFM',
    `X-Auth-Date`: timestamp,
    `X-Auth-Key`: apoc.static.get('podcastkey'),
    `Authorization`: apoc.util.sha1([apoc.static.get('podcastkey')+apoc.static.get('podcastsecret') +timestamp])
    } AS headers
    CALL apoc.load.jsonParams('https://api.podcastindex.org/api/1.0/podcasts/byitunesid?id=' + apoc.text.urlencode($itunesId), headers, '', '') YIELD value
    WITH value.feed AS feed

    // Next we create the (:User)-[:SUBSCRIBES_TO]->(:Podcast) path in the database
    MATCH (u:User {id: $cypherParams.userId})
    MERGE (p:Podcast {itunesId: $itunesId})
    SET p.title       = feed.title,
        p.link        = feed.link,
        p.description = feed.description,
        p.feedURL     = feed.url,
        p.image       = feed.artwork
    MERGE (u)-[:SUBSCRIBES_TO]->(p)
    WITH p

    // Now we parse the podcast XML feed and add its episodes to the database
    // (:Podcast)-[:HAS_EPISODE]->(:Episode)
    CALL apoc.load.xml(p.feedURL) YIELD value
    UNWIND value._children AS foo
    WITH p,[x in foo._children WHERE x._type = \"item\"] AS episodes
    UNWIND episodes AS episode
    WITH p,[x in episode._children WHERE x._type =\"title\"][0]._text AS title,
        [x in episode._children WHERE x._type =\"description\"][0]._text AS summary,
        [x in episode._children WHERE x._type=\"link\"][0]._text AS link,
        [x in episode._children WHERE x._type=\"image\"][0].href AS image,
        [x in episode._children WHERE x._type=\"enclosure\" AND x.type CONTAINS \"audio\" ][0].url AS audio,
        [x in episode._children WHERE x._type=\"pubDate\"][0]._text AS pubDate,
        [x in episode._children WHERE x._type =\"guid\"][0]._text AS guid
    MERGE (e:Episode {id: guid})
    SET e.title = title,
        e.summary = summary,
        e.link = link,
        e.image = image,
        e.audio = audio,
        e.pubDate = dateTime({epochMillis: apoc.date.parse(pubDate, 'ms', 'EEE, dd MMM yyyy HH:mm:ss zzz')})
    MERGE (e)<-[:HAS_EPISODE]-(p)
    RETURN p
    """
  )
}

We'll also want to add the Episode type definition to our GraphQL API:

type Episode {
  id: ID!
  pubDate: DateTime
  summary: String
  title: String
  link: String
  image: String
  audio: String
  podcast: Podcast @relation(name: "HAS_EPISODE", direction: "IN")
}

User Podcast Feed

A typical flow for our application will be:

Show the user a feed of episodes for all the podcasts to which they subscribe, most recent first
The user adds episodes they want to listen to to a playlist
When the user is ready to listen they can view all episodes assigned to each playlist

Let's add a GraphQL query field to show all episodes for each podcast a user subscribes to, showing the most recent episodes first. We'll use the cypherParams.userId value, which will identify the currently authenticated user, to look up that user in the database, traverse to all podcasts they subscribe to, and then traverse to each episode for their subscribed podcasts. Refer to the previous post in this series for details about the cypherParams.userId value and how that is parsed from the auth token.

type Query {
  episodeFeed(first: Int = 10, offset: Int = 0): [Episode]
  @cypher(
    statement: """
    MATCH (u:User {id: $cypherParams.userId})-[:SUBSCRIBES_TO]->(p:Podcast)-[:HAS_EPISODE]->(e:Episode)
    RETURN e ORDER BY e.pubDate DESC SKIP toInteger($offset) LIMIT toInteger($first)
    """
  )
}

First, we'll log in using the username and password we created previously:

This gives us the auth token that when added to the request header will allow us to make authenticated GraphQL requests to the API. Now, let's make an authenticated request to query for our user's podcast episode feed, showing the most recent episodes across all the podcasts I subscribe to:

Playlists

Next, let's introduce the concept of playlists to our API. We want to add individual episodes to one or more playlists, which we should then be able to query for later and play episodes from each playlist. First, we'll add the Playlist type to our GraphQL type definitions:

type Playlist {
  name: String!
  episodes: [Episode] @relation(name: "IN_PLAYLIST", direction: "IN")
}

We want playlists to be private to the currently authenticated user so we'll add the Playlist type to the configuration to be excluded from automatic query and mutations. Instead we'll create @cypher directive fields to access playlists.

const schema = makeAugmentedSchema({
  typeDefs,
  resolvers,
  config: {
    query: {
      exclude: ['PodcastSearchResult', 'AuthToken', 'User', 'Playlist']
    },
    mutation: {
      exclude: ['PodcastSearchResult', 'AuthToken', 'User', 'Playlist']
    }
  }
});

Create Playlist

Since we added the Playlist type to be excluded from the generated queries and mutations during the schema augmentation process we'll need to define the logic for creating and updating playlists using @cypher schema directive fields. Let's add a createPlaylist field to the Mutation type:

type Mutation {
  createPlaylist(name: String!): Playlist
  @cypher(
    statement: """
    MATCH (u:User {id: $cypherParams.userId})
    MERGE (p:Playlist {name: $name})<-[:OWNS]-(u)
    RETURN p
    """
  )
}

Now, we can create a playlist. Let's create a playlist called "jogging" that I'll use to listen to episodes while I'm on jogs around the neighborhood:

Note that the episodes array is empty - we haven't added any episodes to this playlist yet. Let's add that functionality to the GraphQL API now.

Add Episodes To Playlist

To enable users to add episodes to each playlist we'll create a new mutation field called addEpisodeToPlaylist that will take as arguments the name of the playlist and the id of a podcast episode to be added to the playlist. We'll define the logic for this mutation using a @cypher schema directive, ensuring that we match on the authenticated user and make sure they are the owner of the playlist before adding an IN_PLAYLIST relationship connecting the Episode and Playlist nodes.

type Mutation {
  addEpisodeToPlaylist(name: String!, podcastId: ID!): Playlist
  @cypher(
    statement: """
    MATCH (u:User {id: $cypherParams.userId})-[:OWNS]->(p:Playlist {name: $name})
    MATCH (e:Episode {id: $podcastId})
    MERGE (e)-[:IN_PLAYLIST]->(p)
    RETURN p
    """
  )
}

Get Playlist Episodes

Finally, we'll implement a query field to return all playlists a user owns, which can then also be used to return the episodes in each playlist, including episode details.

type Query {
  playlists: [Playlist]
  @cypher(
    statement: """
    MATCH (u:User {id: $cypherParams.userId})-[:OWNS]->(p:Playlist)
    RETURN p
    """
  )
}

DEV Community: William Lyon

Introducing The Spatial Cypher Cheat Sheet

A Resource For Working With Geospatial Data In Neo4j

Introduction To Geospatial Cypher Functions With Neo4j

Intro To Cypher And The Property Graph Data Model

Spatial Point Type

Spatial Cypher Functions

Data Import

Routing With Path Finding Algorithms

Using Neo4j With Python For Geospatial Data

The Neo4j Python Driver

Working With OpenStreetMap Data

Loading A Road Network With OSMNx

Resources

10 Things We Learned In Full Stack GraphQL Book Club

Chapter 1: What Is Full Stack GraphQL?

Chapter 2: Graph Thinking With GraphQL

Chapter 3: Graphs In The Database

Chapter 4: The Neo4j GraphQL Library

Chapter 5: Building User Interfaces With React

Chapter 6: Client Side GraphQL With Apollo Client

Chapter 7: Adding Authentication and Authorization

Chapter 8: Deploying Our Full Stack GraphQL Application

Chapter 9: Advanced GraphQL Considerations

Takeaway #10: Bonus: Low Code GraphQL With The Neo4j GraphQL Toolbox

Download Your Free Copy of Full Stack GraphQL Applications

Spatial Search Functionality With Neo4j

Spatial Cypher Functions And The Point Type

Spatial Search With Neo4j

Radius Distance Search

Cypher Bounding Box Search

Point In Polygon Search

Working With Line Geometries

Routing With Graph Algorithms

Resources

Graph Data Visualization With GraphQL & react-force-graph

Force Directed Layouts & Graph Visualization

Graph Data Visualization With react-force-graph

Using react-force-graph with Next.js

Setting Up Apollo Client In Next.js

Pulling In GraphQL Data

Adding More Nodes On Click

Representing Nodes With Text

Resources

Building A GraphQL Server With Next.js API Routes

Interactive Graph Visualization With Neo4j Bloom

Next.js

create-next-app

Next.js API Routes

Creating A GraphQL Server In A Next.js API Route

Using GraphQL Playground With Apollo Server v3

Using The Neo4j GraphQL Library

Deploying To Vercel

Resources

No Cost Data Scraping With GitHub Actions And Neo4j Aura

A Look At The Data - Lobsters

Neo4j Aura

GitHub Actions And Flat Graph

The Flat Graph GitHub Action

What's Next?

Resources

johnymontana / lobste.rs-graph

Importing Lobste.rs data into Neo4j Aura using GitHub Actions

Getting Started With Next.js and GraphQL Authentication | Building GRANDcast.FM Episode 4

What Is Next.js?

Getting Started With Next.js

GraphQL Data Fetching With Next.js

Making GraphQL Queries With Apollo Client In A Next.js Page

Slight Detour - Setting Up The GraphQL VS Code Extension

Authenticated GraphQL Requests In Next.js

The Sign In Flow

Resources

Parsing And Importing XML With Neo4j: Adding Episodes and Playlists To GRANDcast.FM Podcast App

Working With Podcast Feeds

Parsing XML With Neo4j

Adding Podcast Episodes To The Graph

Adding Episodes When A User Subscribes To A Podcast

User Podcast Feed

Playlists

Create Playlist

Spatial Cypher Functions And The `Point` Type

Graph Data Visualization With `react-force-graph`

Using `react-force-graph` with Next.js

`create-next-app`