ENG-8154: update check_dead_links script and fix broken links (#1682)

Author: carlosabadia

case-studies/ansa.md

@@ -51,7 +51,7 @@ rx.vstack(
 )
 ```
 
-Meet [Ansa](www.ansa.co), a venture capital firm based in New York City that invests in companies from Series A to C. They have invested in companies like Defense Unicorns, Bland, Gradient, and Selector and prior to founding the firm, supported investments in many of the venture-capital industry’s largest outcomes including Crowdstrike, Coinbase, and SurveyMonkey to name a few.
+Meet [Ansa](https://www.ansa.co), a venture capital firm based in New York City that invests in companies from Series A to C. They have invested in companies like Defense Unicorns, Bland, Gradient, and Selector and prior to founding the firm, supported investments in many of the venture-capital industry’s largest outcomes including Crowdstrike, Coinbase, and SurveyMonkey to name a few.
 
 Ryan Sullivan is an investor and oversees the engineering and data science team at Ansa. He finds and supports new investments and is the architect behind Ansa's data-driven sourcing strategy, working closely with the firms’ Managing Partner Marco Demeireles to build the firms’ proprietary sourcing applications and research products.
 

case-studies/sellerx.md

@@ -117,7 +117,7 @@ Below are some of the key Reflex features their app took advantage of:
 
 - Reflex's background events for long running scraping tasks to extract all the data that they needed.
 
-- [AG Grid]({library.tables_and_data_grids.ag_grid.path}), which is built into Reflex, to display the data in a table. AG Grid is a high performance data grid that is used in many fortune 500 companies web applications.
+- [AG Grid](/docs/enterprise/ag-grid), which is built into Reflex, to display the data in a table. AG Grid is a high performance data grid that is used in many fortune 500 companies web applications.
 
 - SSO was a requirement for the app, and Mike used Azure Auth, one of the many options provided by Reflex for authentication.
 

docs/custom-components/command-reference.md

@@ -1,3 +1,7 @@
+```python exec
+from pcweb.pages import docs
+```
+
 # Command Reference
 
 The custom component commands are under `reflex component` subcommand. To see the list of available commands, run `reflex component --help`. To see the manual on a specific command, run `reflex component <command> --help`, for example, `reflex component init --help`.
@@ -81,11 +85,11 @@ The `custom_components` folder is where the actual implementation is. Do not wor
 
 `reflex_google_auth` is the top folder for importable code. The `reflex_google_auth/__init__.py` imports everything from the `reflex_google_auth/google_auth.py`. For the user of the package, the import looks like `from reflex_google_auth import ABC, XYZ`.
 
-`reflex_google_auth/google_auth.py` is prefilled with code example and instructions from the [wrapping react guide]({wrapping_react.overview.path}).
+`reflex_google_auth/google_auth.py` is prefilled with code example and instructions from the [wrapping react guide]({docs.wrapping_react.overview.path}).
 
 ### Demo App Folder
 
-A demo app is generated inside `google_auth_demo` folder with import statements and example usage of the component. This is a regular Reflex app. Go into this directory and start using any reflex commands for testing. The user is encouraged to deploy the demo app, so it can later be included as part of the [Gallery]({gallery.path}).
+A demo app is generated inside `google_auth_demo` folder with import statements and example usage of the component. This is a regular Reflex app. Go into this directory and start using any reflex commands for testing.
 
 ### Help Manual
 

docs/enterprise/mantine/pill.md

@@ -51,7 +51,7 @@ def pill_group_page():
 By itself it does not include any logic, it only renders given children.
 
 ```md alert info
-# For a fully functional out-of-the-box component, consider using [`rxe.mantine.tags_input`]({docs.mantine.tags_input.route}) instead.
+# For a fully functional out-of-the-box component, consider using [`rxe.mantine.tags_input`](/docs/enterprise/mantine/tags-input/) instead.
 ```
 
 ## Example

docs/getting_started/dashboard_tutorial.md

@@ -1606,7 +1606,7 @@ The `radius` prop sets the global radius value for the app that is inherited by
 
 The `accent_color` prop sets the accent color of the app. Check out other options for the accent color [here]({docs.library.other.theme.path}).
 
-To see other props that can be set at the app level check out this [documentation](docs.styling.theming.path)
+To see other props that can be set at the app level check out this [documentation]({docs.styling.theming.path})
 
 ```python
 app = rx.App(

docs/library/data-display/callout.md

@@ -30,7 +30,7 @@ A `callout` is a short message to attract user's attention.
 rx.callout("You will need admin privileges to install and access this application.", icon="info")
 ```
 
-The `icon` prop allows an icon to be passed to the `callout` component. See the [**icon** component for all icons that are available.]({docs.library.data_display.icon.path})
+The `icon` prop allows an icon to be passed to the `callout` component. See the [**icon** component for all icons that are available.](/docs/library/data-display/icon)
 
 ## As alert
 

docs/library/forms/select.md

@@ -181,7 +181,7 @@ def select_form_example():
 
 ### Using Select within a Drawer component
 
-If using within a [Drawer](../../overlay/drawer/) component, set the `position` prop to `"popper"` to ensure the select menu is displayed correctly.
+If using within a [Drawer](/docs/library/overlay/drawer) component, set the `position` prop to `"popper"` to ensure the select menu is displayed correctly.
 
 ```python demo
 rx.drawer.root(

pcweb/pages/gallery/apps.py

@@ -10,7 +10,7 @@
 from pcweb.components.icons import get_icon
 from pcweb.constants import SCREENSHOT_BUCKET
 from pcweb.flexdown import xd2 as xd
-from pcweb.pages.gallery import gallery
+from pcweb.pages import docs
 from pcweb.templates.gallery_app_page import gallery_app_page
 
 GALLERY_APP_SOURCES = [
@@ -73,7 +73,8 @@ def more_posts(current_post: dict) -> rx.Component:
                 selected_posts = other_posts[current_index - 2 : current_index + 1]
 
     for _path, document in selected_posts:
-        posts.append(gallery_app_card(app=document.metadata))
+        if not _path[0].startswith("reflex_build_templates/"):
+            posts.append(gallery_app_card(app=document.metadata))
 
     return rx.el.section(
         rx.box(
@@ -87,7 +88,7 @@ def more_posts(current_post: dict) -> rx.Component:
                     class_name="flex items-center gap-1.5 border-slate-5 bg-slate-1 hover:bg-slate-3 shadow-small px-1.5 py-0.5 border rounded-md w-auto max-w-full text-slate-9 transition-bg cursor-pointer overflow-hidden border-solid",
                 ),
                 underline="none",
-                href=gallery.path,
+                href=docs.getting_started.open_source_templates.path,
             ),
             class_name="flex flex-row items-center justify-between gap-4",
         ),

pcweb/pcweb.py

@@ -87,6 +87,10 @@
 
 # Add redirects
 redirects = [
+    ("/docs/ui", "/docs/ui/overview"),
+    ("/docs/enterprise/react-flow", "/docs/enterprise/react-flow/overview"),
+    ("/docs/advanced-onboarding", "/docs/advanced-onboarding/code-structure"),
+    ("/docs/hosting/cli", "/docs/hosting"),
     ("/docs/enterprise/ag-grid/index", "/docs/enterprise/ag-grid"),
     ("/docs/ai-builder/integrations", "/docs/ai-builder/integrations/overview"),
     ("/docs/ai-builder/app-lifecycle", "/docs/ai-builder/app-lifecycle/general"),

scripts/check_dead_links.py

@@ -45,6 +45,23 @@ def normalize_url(self, url):
             normalized += f"?{parsed.query}"
         return normalized
 
+    def is_image_url(self, url):
+        """Check if URL points to an image file."""
+        image_extensions = {
+            ".webp",
+            ".png",
+            ".jpg",
+            ".jpeg",
+            ".gif",
+            ".svg",
+            ".ico",
+            ".bmp",
+            ".tiff",
+        }
+        parsed = urlparse(url)
+        path_lower = parsed.path.lower()
+        return any(path_lower.endswith(ext) for ext in image_extensions)
+
     def check_link(self, url, source_page):
         """Check if a single link is working."""
         if url in self.checked_links:
@@ -113,7 +130,9 @@ def extract_links(self, html, page_url):
 
             if url:
                 absolute_url = urljoin(page_url, url)
-                if not absolute_url.startswith(("javascript:", "mailto:", "tel:")):
+                if not absolute_url.startswith(
+                    ("javascript:", "mailto:", "tel:", "data:")
+                ):
                     links.append(absolute_url)
 
         return links
@@ -139,9 +158,10 @@ def crawl_page(self, url):
             links = self.extract_links(response.text, url)
 
             for link in links:
-                self.check_link(link, url)
+                # Only check internal links and skip images
+                if self.is_internal_url(link) and not self.is_image_url(link):
+                    self.check_link(link, url)
 
-                if self.is_internal_url(link):
                     normalized = self.normalize_url(link)
                     if normalized not in self.visited_pages:
                         self.pages_to_visit.append(normalized)