Command Palette - Result relevance

[Techywarrior]

Microsoft PowerToys version

0.95.0

Installation method

PowerToys auto-update

Area(s) with issue?

Command Palette

Steps to reproduce

The updated fuzzy search seems to have worse relevancy then before, at least in regards to system commands. This seems to change every few versions and, for me, makes it not worth using. The relevancy threshold also seems way too low as most of the results I get at best have the individual letters of the search term spread between the title and description. Honestly not even sure why the description has any relevancy but that might just be a personal preference. But I can't imagine anyone wanting apps to show before other results that have 100% title matching.

✔️ Expected Behavior

When typing a system command, such as sleep I expect the first result to be the system command Sleep and not getting 10+ app results, none having the letters sleep in even remotely the correct order (at best they have the individual letters across the title and description) before the system command that 100% matches the search term.

❌ Actual Behavior

The command that I actually searched for is after about 10 pretty much irrelevant app results instead of being the first match (since it 100% matches the search term).

Additional Information

No response

Other Software

No response

[jiripolasek]

Hi @Techywarrior,

Thanks for your feedback! The current setup is only temporary. Sleep is currently a "fallback command" and stays unranked, always showing up at the end of the list (see #41959 for more details in the PR). However, a feature for user-defined ranking is planned for this iteration as #38288.

[Techywarrior]

Thanks for the response. After reading #38288 I believe that would resolve my issue although it will take manual tweaking to get right.

I'm not sure on the reasoning that these system commands are a "fallback command" when there is specifically an extension to enable system commands. That aside it sounds like I just need to eagerly await for user defined rankings (and probably downgrade until then 🙁)

[michaeljolley]

I'm not sure on the reasoning that these system commands are a "fallback command" when there is specifically an extension to enable system commands.

@Techywarrior, that's solid feedback. Let me give a high-level of how top-level commands (TLC) and fallback commands (FBC) work and why they're currently unranked.

When you type something, we are using a fuzzy search to look through TLC to match them. Exact hits on name score highest, followed by starts with, contains, etc.

Next would be TLC that match the description, again with varying weights for exact, starts with, contains, etc.

Next is the fuzzy part you noticed. Can we find the letters you typed (let's say "C-O-P-I-L") in that specific order in the title or description. That's the "fuzzy" bit. Weights here vary and are less than the ones I mentioned above.

With FBC, we send your search string to them because they don't necessarily have a title/description. For instance, the FBC for the Settings extension takes that string and does its own fuzzy search for settings and then updates it's FBC title/description based on its results (if there were any).

Of course, the more extensions (and therefore FBC) you have, the slower the search results will be and the more they will "jump around" as new FBC propagate their new title/description.

So, to solve this, we're taking a few steps that will drop over 0.95 and coming releases:

1. Un-rank all FBC except for two that are "whitelisted" (Calculator & Run). This will cause all non-whitelisted FBC to display at the bottom of the results. Not ideal but makes search much more responsive until the full solution is fully baked.
2. Allow users to weight/rank FBC themselves. This will let you specify that you want Settings to show higher than say Web Search.
3. Allow users to specify that FBC should display in the top-level (global) list. This would allow you to specify that you want Settings to essentially added to the whitelist to be scored and ranked in the main result set.

Of course, you'd want to do that selectively because the more FBC that are included in the global results, the slower your search will become.

It's important to note that Command Palette is super-extensible. That's one of the defining differences from PT Run. But with that power comes great responsibility. 😄

If an extension developer has a fallback that performs its search and subsequent update of title/description slowly, it may affect the experience for the user. (i.e. you get search results back, but then "bang!" out of nowhere another item jumps into the list when that extensions FBC finally gets updated.) Indexer is a good example of this. It can search files on your computer, but it takes a bit. That "bit" can make it feel like CmdPal is slow, when it's actually the extension.

Hopefully that provided a little insight into the blackhole that is my work around fallbacks and CmdPal performance at the moment.

Cheers!

[Techywarrior]

@michaeljolley Wow, first let me say I think it's pretty awesome how you guys are responding to this issue. I've definitely seen projects where there would have just been "working as intended. Closed" response.

I understand the explanation, and can see how the need for this overall change you are making came about, but wondering if for an extension like system commands, which should have a finite and slow changing list of, well commands, would it not be possible to create a cache of the commands and descriptions (if that can be pulled from the system)? While I am a developer myself, I will admit that the world I live in is not as complex as building something to interact with so many low level parts of a large OS.

the FBC for the Settings extension takes that string and does its own fuzzy search for settings and then updates it's FBC title/description based on its results (if there were any)

Since the fallback search does find/generate titles and descriptions it should be possible to cache that to at least make the cache results behave more like the TLC. I don't think that would work with every type of extension though since something like file search couldn't realistically cache results in time. Even in my simplified scenario where I am only looking at the system commands I'm not sure how to handle if/when new ones are added. For this it's probably less of a problem since you could periodically re-cache it (maybe as a background task on app load) since new system commands would be part of an OS update.

[michaeljolley]

Solid idea. In fact, the Settings extension is a bad example because those settings are essentially hard-coded. It's so fast that it likely wouldn't affect performance to have it whitelisted. We probably went overkill in turning off extensions. Maybe we'll add it for the next release.

[kvnxiao]

To add to this from the experience on my side, I typically access the "Empty Recycle Bin" option frequently, pre 0.95. This is a regression for my workflow, and I had to temporarily add an alias to the system commands as a non-perfect workaround.