Imagine this - you rely on data that you download every day from some system to your own. That requires a trip to the server asking for information, and then a trip back with the payload we requested.
This seems pretty fast since the internet is fast. But we also know the programming concept DRY (Don't Repeat Yourself). So, can we apply this principle to how we handle the scenario described above, creating something like DRD (Don't Repeat Data)?
Well, yes. There is something to handle this, and it's called — Zero Copy.
What is Zero Copy?
As the name suggests, you are copying zero data, and yet, you are getting it on your system.
How is this possible? If you think about it, you'll probably come to the conclusion that we are just opening a window. The data is just out there to be looked at by those who are allowed to. There's no need to bring the same data to different people's windows; we're just keeping the data in one place and making it available to anyone who needs it.
What does this mean for ServiceNow?
When it comes to Operations Management—dealing with data fetched from different databases (like monitoring data from Datadog or Dynatrace, ERP data from SAP or Workday, or cloud platforms like Snowflake, AWS, or Azure)—copying that data has traditionally been a hassle.
We were reliant on sometimes complex ETL (Extract, Transform, Load) pipelines or massive data extracts. This complicated the whole process, consumed a lot of time, and required careful checking of data pre- and post-migration.
So how exactly does Zero Copy help us here?
Virtual Data Fabric Tables.
Instead of copying data extracted from other tools, ServiceNow queries the exact data that is requested. It temporarily holds that data in memory for the user to interact with. During that time, the user can leverage that data for various use cases as required—and once they are done, it's gone.
So, what exactly are the benefits of Zero Copy?!
- No need for data duplication on the destination.
- No need for data syncing between source and destination.
- Overhead removed for security on the destination system (since the data doesn't actually live there).
The Catch: What are the Cons of Zero Copy?
- Performance and Latency: Since you fetch data on demand, you are entirely at the mercy of network speeds and the external system's response time. If the source system takes three seconds to process, your user waits three seconds.
- Complete Dependency on Uptime: With traditional ETL, if the external system goes down, you still have the last copied batch. With Zero Copy, if the source system crashes or loses network connectivity, you have no local backup to fall back on.
- Increased Load on the Source System: Instead of one massive overnight extract, every single time a user looks at the data, a live query fires off. This can cause performance bottlenecks on the source's end.
- API Rate Limits and Costs: Many modern cloud platforms charge by the API call or compute usage. Constant live queries can quickly rack up expensive API costs or trigger rate-limiting blocks.
- Limitations on Complex Reporting: Because the data isn't stored in your database locally, you cannot index it. Trying to do complex operations—like joining external data with local tables or generating massive aggregated reports—can be highly inefficient or simply not allowed.
When Should You Actually Use It?
If Zero Copy has performance and dependency drawbacks, why use it at all? The golden rule is: You use Zero Copy when the cost, risk, or time of moving the data outweighs the benefits of storing it locally.
Here are the core scenarios where Zero Copy absolutely crushes traditional data copying:
- Handling Highly Sensitive Data (Compliance): Instead of doubling your security risk by copying PII or HIPAA data into two databases, your users just look through the "window." When they close it, the data vanishes.
- Real-Time Troubleshooting (IT Ops): When a server goes down, you need to know what's happening right now. Zero Copy queries monitoring tools in that exact millisecond, preventing you from relying on stale, 15-minute-old synced data.
- Occasional "Point-in-Time" Lookups: If delivery team occasionally needs to check a budget in SAP, don't sync the entire company's financial database daily. A quick API call grabs that one specific number exactly when needed without wasting resources.
- Working with Massive Datasets: If you have petabytes of logs in Snowflake, moving them across the internet will bottleneck your network. Let Snowflake do the heavy computing, and use Zero Copy to bring just the tiny, summarized result into your system.
Top comments (0)