Scope of this project vs wikibase.world

[addshore]

Hey hey,

It looks like this project is essentially a new version of the wikibase-registry / wikibase.world, except it's not using and wikibase as the data store etc and doesn't yet house as many sites.

Is there some documentaiton on what the scope of this thinng is / the requirements in terms of numbers to be collected that might be useful? (So that I can look at just adding them all to wikibase.world)

[addshore]

Pinging @RickiJay-WMDE @rti @darthmon @jonamar as the contributors to this repo
(Not sure these issues get looked at considering #52 is still open too)

[jonamar]

Hi @addshore thanks so much for asking. And a big apology for taking so long to respond to you.

I know that efforts to collect metrics about Wikibase instances can raise important questions, so I wanted to be intentional and transparent about our goals and approach.

How Our Internal Metrics Project Differs from and Compliments Wikibase.World

1. Wikibase.World is a voluntary registry, but participation takes effort
   The registry gives Wikibase users a way to declare their instance and share metadata—but doing so takes time, awareness, and manual effort. As a result, many projects that would like to be visible aren’t yet represented or are only briefly represented. Our project builds on this intent by (working towards) offering an easier opt-in path, richer data, and a clear consent process.

2. Our goal is to support public visibility, not just internal insights
   While this began as a way for WMDE to better understand self-hosted Wikibase usage, our long-term aim is to connect the data back out for public benefit—most likely through Wikibase.World, but potentially via other avenues too.

3. This is part of a larger effort to serve the community
   When I first joined, it often felt like we were flying blind—we lacked context on who was using Wikibase, how they were using it, and what they needed to succeed. This project, along with ongoing user research and community engagement, is part of a broader effort to better understand and support Wikibase users. As well as a tool that provides us metrics to advocate for the value of the self-hosted community in internal resourcing conversations.

We’re still in the early stages. Our next big steps are to create a lightweight, opt-in, way for instances to be listed in our directory as well as listed in Wikibase.World (or other public directories) without requiring extra manual effort.

We really welcome feedback, questions, or suggestions to make sure this work reflects our shared values.

[addshore]

but participation takes effort

Not entirely true, as the registry automatically ingests

1. All wikibase cloud sites
2. Sites discovered via some basic google searches
3. Sites found in other repositories (including wikibase-metadata)

I should add wikiapiary.com › wiki › Main_Page for example too
Not sure if you know of any other good sources of finding wikibases in the present day?
But I am on the whole all for easier discoverability helped by some wmde powered opt in thingy!

Our goal is to support public visibility, not just internal insights
While this began as a way for WMDE to better understand self-hosted Wikibase usage, our long-term aim is to connect the data back out for public benefit—most likely through Wikibase.World, but potentially via other avenues too.

Sounds good, and essentially already in done as wikibase.world pulls sites at least from wikibase metadata.

So far when looking it didnt make any sense to pull any additional data from it, but the wikibase metadata ID itself is stored as a statement for backlinks

As well as a tool that provides us metrics to advocate for the value of the self-hosted community in internal resourcing conversations.

One thing that would be great would be if the suite team could add any way of showing that a site is powered by suite.
This could be something added to the mediawiki version information? perhaps a different API or site prop exposed that would say that the perticular install is powered by a suite build of wikibase.git, and or mor specifically a docker image of a certian version?

This would make it much easier to give a clear picture of the ecosysytem.

[jonamar]

great feedback @addshore thanks for that. and I was not aware that wikibase.world already ingests data from wikibase metadata. are you operating an external service for the automatic ingestion? would love to hear more about how that works right now.

great call too to make it clearer that instances are based on suite's stack. I'll add that for consideration for the discovery feature.

[addshore]

All at https://github.com/addshore/wikibase.world

There are 4 scripts in https://github.com/addshore/wikibase.world/tree/main/cmd
These:

• Import wikibases from wikibase.cloud
• Import from wikibase metadata
• Import from a crude google search
  And the last one loops through all sites in wikibase.world extracting and updating information about them by scraping

The plan was I would just run these scripts on github actions every week
However, at this point I just run them myself every few weeks.