[Diagram Editor] Frontend progress visualization by ArizmendiWan · Pull Request #194 · open-rmf/crossflow

ArizmendiWan · 2026-05-16T20:43:52Z

Adds the first frontend-visible debugging slice for the diagram editor.

It adds live operation-start visualization in the graph, exposes the existing backend debug WebSocket through the REST API client, and moves Run/Debug controls into the persistent right side panel so runtime status no longer blocks the diagram canvas.

It also adds a small delayed calculator example so the progress visualization is easy to verify manually.

Changes

Highlight the currently executing node and retain visited nodes during debug runs.
Move Run/Debug UI from popover into the right side panel under a Run tab.
Add optional wsDebugWorkflow support to the frontend API client.
Add DebugSession handling for backend operationStarted and finish messages.
Add delay_ms to the calculator example catalog.
Add carry_object_progress.json as a latency-focused progress demo.

Demo Video

Screen.Recording.2026-05-21.at.20.34.02.mov

GenAI Use

We follow OSRA's policy on GenAI tools

I used a GenAI tool in this PR.
I did not use GenAI

Generated-by: Codex

Signed-off-by: ArizmendiWan <2311602492@qq.com>

…progress Signed-off-by: ArizmendiWan <2311602492@qq.com> # Conflicts: # diagram-editor/dist.tar.gz # diagram-editor/frontend/command-panel.tsx # diagram-editor/frontend/diagram-editor.tsx # diagram-editor/frontend/diagram-properties-drawer.tsx # diagram-editor/frontend/run-button.tsx

aaronchongth

Thanks for hammering this out! Introspection and debugging will definitely help users more quickly test workflows.

Before I go through the review more thorough, I have some high-level comments about this feature. Some background reference, #86. Just to clarify, I believe these features regarding tracing events were called/referenced as "debug" , but it is part of introspection tools and should be provided out-of-the-box, and not just in debug mode for crossflow. Especially since they provide incredible value to users while incurring very little overhead, I'd suggest that UX-wise, we have the checkbox turned on by default beside the "Run" button in the Run panel, that only toggles the visualization, while all the tracing events are still provided. Let me know what you think or if I had the wrong understanding of this implementation.

I was able to follow the readme and run the example, but it would be cool to also include screen recordings of the new features and how to interact with them on PRs.

Signed-off-by: ArizmendiWan <2311602492@qq.com>

ArizmendiWan · 2026-05-22T01:43:49Z

Thanks for hammering this out! Introspection and debugging will definitely help users more quickly test workflows.

Before I go through the review more thorough, I have some high-level comments about this feature. Some background reference, #86. Just to clarify, I believe these features regarding tracing events were called/referenced as "debug" , but it is part of introspection tools and should be provided out-of-the-box, and not just in debug mode for crossflow. Especially since they provide incredible value to users while incurring very little overhead, I'd suggest that UX-wise, we have the checkbox turned on by default beside the "Run" button in the Run panel, that only toggles the visualization, while all the tracing events are still provided. Let me know what you think or if I had the wrong understanding of this implementation.

I was able to follow the readme and run the example, but it would be cool to also include screen recordings of the new features and how to interact with them on PRs.

I think you’re right about introspection. Progress visualization is broadly useful for monitoring workflows and intuitive understanding of execution, not only for debugging.

I originally approached this from the diagram editor debugging use case, where progress visualization felt necessary to make debugging usable, which is why the current commits use that naming and flow. I agree it would be better to reposition this as a normal introspection/progress feature. I’ll update the PR in that direction.

edit: I just realized that the progress visualization is entirely built on top of debugging backend, which passes the status information of nodes. If we want to make the introspection default, we may have to make debug default too. IMO the overhead is acceptable.

Signed-off-by: ArizmendiWan <2311602492@qq.com>

aaronchongth

Overall, I think the plumbing is working as expected, I've left some comments about the feature as well as the UX.

edit: I just realized that the progress visualization is entirely built on top of debugging backend, which passes the status information of nodes. If we want to make the introspection default, we may have to make debug default too. IMO the overhead is acceptable.

You're right about that. I also had the discussion with @koonpeng and in the early stages, it was considered to only be used for debugging purposes, but with a greater focus on building and running workflows from the UX perspective, I believe this should all be under the router feature. And any mention of "debug" should be renamed to either along the lines of progress visualization or just wsSession. Users from the frontend, can choose to enable the visualization or not.

aaronchongth · 2026-06-08T03:40:49Z

+        pass the input through unchanged. This is useful for making example \
+        workflow progress visible in the diagram editor debugger.";
+    let delay_examples = [
+        ConfigExample::new("Wait for the default 750 ms.", json!(null)),


Is there a reason 750ms was picked, as opposed to the usual more commonly used 1000ms?

There is no specific reason. This is an arbitrary number. I will change the default number to 1000 in the following commit.

aaronchongth · 2026-06-08T03:41:33Z


+    let delay_description = "Wait for a configured number of milliseconds, then \
+        pass the input through unchanged. This is useful for making example \
+        workflow progress visible in the diagram editor debugger.";


Mention in the description that,

This node is only for testing the progress visualization (also mention the example workflow json), and it should not be used in other workflows to naively introduce delays for async purposes.

what the default value is, if not configured

aaronchongth · 2026-06-08T06:58:10Z

+            <Tab
+              value="run"
+              label="Run"
+              icon={<MaterialSymbol symbol="play_arrow" />}


I think we should still have the properties tab as the first tab to be displayed by default, this would be very helpful whenever users load a new diagram in, and is able to see the description and example input requests.

aaronchongth · 2026-06-08T07:20:33Z

+        outline: debugLatest
+          ? `2px solid ${theme.palette.success.main}`
+          : debugVisited
+            ? `1px solid ${alpha(theme.palette.info.main, 0.35)}`


Suggested change

? `1px solid ${alpha(theme.palette.info.main, 0.35)}`

? `2px solid ${alpha(theme.palette.info.main, 0.35)}`

the outline for visited nodes is is same as the default outline, 1px, changing it to 2px will at least make a visible difference

aaronchongth · 2026-06-08T07:28:06Z

+        outline: debugLatest
+          ? `2px solid ${theme.palette.success.main}`


so far it looks like we are only display the success color on the latest node that has reported it has started. We should have the success color on all ongoing active nodes, and only extinguish it back to 2px info (visited), once those nodes have reported that it has finished.

Example of this issue, by adding a fork clone in the example json of this PR, and repeating another branch of delays, we see that only the most recent reported delay node is active on each step of the separate branch, when they both should be lit up

Screencast.From.2026-06-08.15-26-46.mp4

ArizmendiWan added 4 commits May 15, 2026 15:19

Add frontend debug progress visualization

035661c

Signed-off-by: ArizmendiWan <2311602492@qq.com>

Move run controls into side panel

20b94fc

Signed-off-by: ArizmendiWan <2311602492@qq.com>

Add progress demo for frontend debugging

17f83cd

Signed-off-by: ArizmendiWan <2311602492@qq.com>

Remove memory file and add comment to delay

ba26ccb

Signed-off-by: ArizmendiWan <2311602492@qq.com>

mxgrey added this to PMC Board May 16, 2026

github-project-automation Bot moved this to Inbox in PMC Board May 16, 2026

mxgrey requested a review from aaronchongth May 19, 2026 01:30

mxgrey moved this from Inbox to In Review in PMC Board May 19, 2026

aaronchongth requested changes May 21, 2026

View reviewed changes

Comment thread examples/diagram/calculator/README.md Outdated

Update README.md to correct cargo run command

5d9715c

Signed-off-by: ArizmendiWan <2311602492@qq.com>

ArizmendiWan requested a review from aaronchongth May 22, 2026 02:28

ArizmendiWan added 2 commits May 22, 2026 00:23

Make progress visualization default with optional disabling

5e57926

Signed-off-by: ArizmendiWan <2311602492@qq.com>

Merge 'upstream/main'

103f844

Signed-off-by: ArizmendiWan <2311602492@qq.com>

ArizmendiWan mentioned this pull request May 26, 2026

Merge frontend progress visualization into python-demo #202

Merged

2 tasks

aaronchongth requested changes Jun 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Diagram Editor] Frontend progress visualization#194

[Diagram Editor] Frontend progress visualization#194
ArizmendiWan wants to merge 8 commits into
open-rmf:mainfrom
ArizmendiWan:frontend-debugging-progress

ArizmendiWan commented May 16, 2026 •

edited

Loading

Uh oh!

aaronchongth left a comment

Uh oh!

Uh oh!

ArizmendiWan commented May 22, 2026 •

edited

Loading

Uh oh!

aaronchongth left a comment

Uh oh!

aaronchongth Jun 8, 2026

Uh oh!

ArizmendiWan Jun 8, 2026

Uh oh!

aaronchongth Jun 8, 2026

Uh oh!

aaronchongth Jun 8, 2026

Uh oh!

aaronchongth Jun 8, 2026

Uh oh!

aaronchongth Jun 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	? `1px solid ${alpha(theme.palette.info.main, 0.35)}`
	? `2px solid ${alpha(theme.palette.info.main, 0.35)}`

		outline: debugLatest
		? `2px solid ${theme.palette.success.main}`

Conversation

ArizmendiWan commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Demo Video

GenAI Use

Uh oh!

aaronchongth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ArizmendiWan commented May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aaronchongth left a comment

Choose a reason for hiding this comment

Uh oh!

aaronchongth Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

ArizmendiWan Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

aaronchongth Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

aaronchongth Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

aaronchongth Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

aaronchongth Jun 8, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ArizmendiWan commented May 16, 2026 •

edited

Loading

ArizmendiWan commented May 22, 2026 •

edited

Loading