Michael O'Brien

Posted on Nov 3, 2021 • Originally published at sensedeep.com

SenseDeep DynamoDB Data Browser

#aws #serverless #dynamodb #singletable

The SenseDeep Data browser is a DynamoDB data browser and editor that can be used to query, manage and modify your DynamoDB data.

Developers are adopting DynamoDB single-table design patterns as the preferred design model where all application data is packed into a single-table. Combing disparate items with different attributes into one table can make browsing, navigating, organizing and viewing data obscure and difficult.

Managing single-table data and performance can often feel like you are peering at Assembly Language as it is hard to decode overridden keys and attributes manually. A new generation of tools is required.

SenseDeep can understand your single-table designs and make sense of your data and present items as intuitive application entities instead of raw data. The SenseDeep data browser is single-table "aware". This means SenseDeep can transform raw data items to present as application entities and fields.

Single-Table Data Browser

The SenseDeep data browser can query, manage and modify your table data. It supports browsing by scan, query or by single-table entities.

While the data browser can be used to browse and manage any DynamoDB table, it is turbo-charged when a schema describing your data is applied.

A scheme describes your application entities and their attributes. It specifies exactly how your data should be interpreted and what data is valid to store in the database table. You can import a schema from a JSON file (such as a OneTable schema) or you can define a schema using the SenseDeep single-table designer.

When armed with a schema, SenseDeep is able to "understand" your data and organize and present data items as application entities instead of raw, encoded DynamoDB items. The SenseDeep data browser intelligently displays and formats data according to your single-table schema. This not only makes your encoded single-table data easier to understand, but it guides and validates your changes and prevents schema-breaking and application breaking data changes.

The data browser supports DynamoDB scans, native queries or queries by application entity. Queries by application entity are normal DynamoDB queries but where SenseDeep understands which attributes are required to select and filter your data items.

Scanning

When scanning, select the index to scan and provide optional additional filtering attributes. Attributes can be combined using AND or OR operators. SenseDeep translates these instructions into DynamoDB scan commands. You can click on the Command button to see the generated DynamoDB command.

Querying

When querying, specify the index and partition key with optional sort key value. Additional filters may also be specified.

You can use sort key operations to select a single matching item with "Equal" or one or more items with the other Sort Key Operators from the pull down menu.

Query By Entity

When querying by Entity, you select the desired entity model and SenseDeep will select the appropriate attribute filters for that model.

SenseDeep understands the schema for the selected model and what are the required attributes to retrieve specific entity items. Sensedeep will guide you with the list of attributes for that entity model.

Saving Queries

Queries may be saved to your database table where they are persisted in the schema. The schema, saved in the database table, contains the entity definitions, saved queries and modeling data. This makes your table self-describing for 3rd party tools.

You can load and delete queries using the Queries button.

Data Visualization

SenseDeep groups, organizes and color-codes query results for maximum clarity.

Cells are color coded:

A cell is green if it can be edited. Click on the cell to modify the value (inline).
A cell is pink if the cells value is derived from a computed value template using other attributes as ingredients.
A cell has a wavy gray background if it is not relevant for this item (as defined by the schema).
A cell has a blue underline hot-link if it is a reference to another item and can be clicked to jump to the referenced item.

Columns are ordered with the index partition and sort key first, followed by the schema entity type attribute (if defined). After that, columns for all items are ordered alphabetically. If an item has many attributes, click the edit icon to display the item attributes vertically.

Hot Linked Items

Databases like DynamoDB have relationships between items. Just because it is a NoSQL database, does not mean there are no relationships. It just means there is little support in terms of joining tables, enforced foreign keys and data relationship integrity.

SenseDeep and OneTable bridge this gap by defining relationships between entities in the schema. SenseDeep will interpret these relationship links and highlight those in the browser with blue underlining.

You can quickly traverse related items by clicking on a blue hot link in the query results.

SenseDeep defines relationships between entity items using the schema "Reference" field property. This property specifies that a field refers to another entity item and which attributes are required to uniquely identify the target item.

Hot Link References

The format of the "Reference" field is:

Model:index:attribute-1=source-attribute-1,...

For example, the following Reference defines a link to an Account item:

Account:primary:id=accountId

This means: select the Account entity using the "primary" index and the "Account.id" attribute using the value of "accountId" from this item.

If you control-click (or Cmd-Click) on a hot-link SenseDeep uses the reference and determines the relevant query to locate the item and automatically fills the index, key and filter attributes accordingly. Queries are entered into your browser history so you can click the browser Back button to easily jump backwards to the original item.

You can also save these hot-linked queries using Queries -> Save.

Show Templates

An essential single-table design technique is decouple your keys from regular data attributes. This greatly enhances your ability to evolve your DynamoDB data going forward. SenseDeep and OneTable support this technique via value templates where key values are composed using templates that combine the values of other attributes at runtime.

It is often useful to view the value templates vs the calculated key values. The Templates toggle above the table switches between displaying the template values vs the actual data values.

Similarly, there may be some attributes that are designated as hidden. Changing the hidden toggle will display or hide these attributes.

Context Menu

In any data cell, you can right click to display the context menu for additional command options.

The options are:

Copy to copy the current item to the clipboard in JSON format.
Design Schema will jump to the SenseDeep single-table designer for the entity on which the current item is based.
Edit will open the editor slide-out panel for easy editing of the whole item.
Follow Reference behaves the same as Cmd-Click to follow a hot link reference.
Generate Mock Data option can be used when developing to generate sample data, such as email addresses or phone numbers.

Modifying Data

You can modify data inline by clicking on any green cell. Once modified, the Save button will be displayed above the items to persist the changes to the table.

To add a new item, click the Add Item button. You can select the desired entity model (if a schema is present) and it will intelligently prompt you for the appropriate attributes.

Modify Panel

You can also edit by clicking on the Edit pencil icon at the start of each item. This will display a slide out editor panel that will display only the attributes of the item, organized vertically.

The red cells have their value derived from other attributes and cannot be edited. You can modify the value template in the schema via the SenseDeep single-table designer.

If you change the partition key values either directly or indirectly be changing attributes that are used in a key value template, SenseDeep will atomically remove the old item and create the new item via a DynamoDB transaction.

When you click save, the changes are accepted, but you still must click the Save button on the query page to persist results to the table.

Import / Export

No man is an island and your data must be easy to export or import. SenseDeep provides an "Export to AWS Workbench" option. This will export your schema and data items into a WorkBench model. This model can also be imported by the Dynobase app.

When exporting, you should limit the amount of data you export. WorkBench models are for development and are designed for limited data sets. You can also export in a JSON backup format.

When importing, you can import a model to an existing (empty) table, or you can dynamically create a new table to hold the imported model.

Summary

Gaining insight into single-table design patterns is the new frontier for DynamoDB and the SenseDeep DynamoDB Studio is the start of a new wave of tools to elevate and transform DynamoDB development.

Previously, single-table design with DynamoDB was a black box and it was difficult to peer inside and see how the components of your apps are operating and interacting. Now, SenseDeep can understand your data schema and can transform raw DynamoDB data to highlight your application entities and relationships and transform your effectiveness with DynamoDB.

SenseDeep includes a table manager, data item browser, single-table designer, provisioning planner, database migration manager and in-depth table metrics — all of which are single-table aware.

More?

Try the SenseDeep DynamoDB studio with a free developer license at SenseDeep App or learn more at https://www.sensedeep.com.

About SenseDeep

SenseDeep is an observability platform for AWS developers to accelerate the delivery and maintenance of serverless applications.

SenseDeep helps developers through the entire lifecycle to create observable, reliable and maintainable apps via an integrated serverless developer studio that includes deep insights into how your apps are performing.

To try SenseDeep, navigate your browser to: https://app.sensedeep.com.

To learn more about SenseDeep please see: https://www.sensedeep.com/product.

Please let us know what you think, we thrive on feedback. dev@sensedeep.com.

DEV Community