DEV Community

Crawler Bros
Crawler Bros

Posted on

Uncover Your Website's Hidden SEO Gold with a Link Graph

Beyond the Sitemap: Unlocking Your Website's True Structure

As an SEO professional, content marketer, or web developer, you know the power of internal linking. It’s not just about passing “link juice”; it's about guiding users, establishing topical authority, and helping search engine bots discover every valuable page on your site. But how well do you really understand your website’s internal link architecture?

You might have a general idea, or rely on a standard sitemap. But sitemaps are flat lists; they don't show the intricate, interwoven relationships between your pages. They don't expose orphaned content, identify critical pages with too few internal links, or highlight redundant paths that confuse search engines and users alike.

Imagine being able to see your website's internal linking structure as a living, breathing network. A visual map where each page is a node and every link is a connection. This isn't just a hypothetical dream; it's a powerful reality that can transform your SEO strategy.

Introducing the Website Links Graph Generator: Your Blueprint for SEO Success

Apify's Website Links Graph Generator (actor web-link-graph-visualizer) is designed precisely for this purpose. It crawls your specified URL, analyzes all internal and external links, and generates a stunning, interactive graph visualizing these connections. Beyond the visual, it also provides structured data that empowers deeper analysis.

Let's explore how this actor helps you tackle critical SEO and website management challenges:

1. Identify Orphaned Pages and Boost Discoverability

One of the most common SEO pitfalls is having valuable content that's not linked to from anywhere else on your site (or at least, very few places). These "orphaned pages" are hard for both users and search engine crawlers to find, significantly diminishing their SEO potential.

With the Website Links Graph Generator, you can easily spot these isolated nodes. Pages with few or no incoming links will stand out, allowing you to strategically add internal links from relevant, high-authority pages. This dramatically improves their discoverability and helps them rank higher.

2. Optimize Internal Link Flow and Authority Distribution

Think of your website's authority as a river flowing through your pages. Internal links are the channels. You want this authority to flow efficiently to your most important landing pages, product pages, or cornerstone content.

By visualizing the link graph, you can quickly identify:

  • Key Hub Pages: Pages with many outgoing and incoming links often represent central topics or navigation points.
  • Weak Points: Pages that are important but receive very few internal links might need more attention.
  • Link Silos: Are certain sections of your site overly isolated, preventing authority from flowing across relevant topics?

You can configure the actor's maxDepth input to control how deep the crawl goes, ensuring you capture the full scope of your internal linking. By understanding these patterns, you can strategically adjust your internal linking to direct more authority to your most crucial content, improving their ranking potential.

3. Pinpoint and Fix Broken Links (Internal & External)

Broken links (404 errors) are detrimental to user experience and SEO. They waste crawl budget, signal a poorly maintained site to search engines, and frustrate visitors. While many tools check for broken links, the Website Links Graph Generator provides a visual context for these issues.

When the actor runs, it not only creates graph.png (your visual map) and graph.json (the structured data), but it also produces a detailed dataset of all crawled pages. This dataset includes information about the links found on each page, including their status. You can filter this dataset to quickly identify 404s and then use the graph to see which pages are linking to them. This contextual understanding makes fixing broken links much more efficient.

The includeExternal option is also incredibly useful here. Even if you don't want to crawl external sites, checking this box will display external links in your graph. This helps you monitor the quality and status of your outbound links, ensuring you're not linking to broken or spammy external resources.

4. Enhance User Experience and Website Navigation

A well-structured website isn't just for bots; it's paramount for human users. A clear, intuitive internal linking structure helps visitors find what they're looking for, reduces bounce rates, and encourages deeper engagement with your content.

By seeing the actual paths users can take (via internal links), you can evaluate if your navigation is logical and efficient. Are there dead ends? Are key pieces of information buried too deep? The visual graph helps you empathize with your users' journey and make data-driven decisions to improve information architecture. You can even choose nodeLabels like 'page title' or 'URL' to make the graph instantly understandable.

How to Use the Website Links Graph Generator

Ready to unveil your website's linking structure? Here’s a quick guide:

  1. Find the Actor: Navigate to the Apify Store and search for "Website Links Graph Generator" or web-link-graph-visualizer.
  2. Start a Run: Click the "Try for free" or "Start run" button.
  3. Enter Your Start URL: In the startUrl field, enter the full URL of the page where you want the crawl to begin (e.g., https://www.yourdomain.com/).
  4. Define Boundaries (Optional but Recommended): To ensure the actor only crawls your domain, use the boundaryRegex field. For example, if your domain is yourdomain.com, you'd enter https?://[^/]*yourdomain\.com/.*. This tells the actor to only crawl URLs matching that pattern.
  5. Set Max Depth & Pages (Optional): Control the maxDepth (e.g., 3 for a shallow crawl, higher for deeper analysis) and maxPages to manage the scope of your crawl.
  6. Choose Layout and Labels: Select your preferred graphLayout (e.g., FruchtermanReingold, Circle) and nodeLabels (e.g., url, page title) for the visual output.
  7. Run and Review: Click "Start" and let the actor do its work. Once complete, you'll find the graph.png and graph.json files in the Key-Value Store, along with a dataset of all crawled pages for detailed analysis.

Transform Your SEO with Visual Insights

The Website Links Graph Generator isn't just another SEO tool; it's a new lens through which to view and understand your website. It transforms abstract data into actionable visual insights, empowering you to make smarter decisions about your internal linking strategy, improve user experience, and ultimately, climb higher in search engine rankings. Stop guessing about your website's structure and start seeing it clearly.


Ready to try it yourself? Run *Website Links Graph Generator** on the Apify Store -- no setup required.*

Top comments (0)