Use probes as targets

[jimaek]

The basic use-case:

Allow an organization to run probes in their DCs and then use them as targets when running tests from any public GP probe.
Assume they don't know the IP addresses of each of the probes. Meaning the would want to run something like
globalping ping probe3-de-org12 from germany+orange

Alternatively we could instead do public "vantage points" or whatever the name would be. Allowing all users to mark any of their probes as such, which would then:

1. Mark the probe in the API
2. Show the probe in a public directory we would make listing all of them
3. Apply the "magic" logic to targets? e.g. ping poland+hetzner from germany+hetzner

Needs some discussion

[MartinKolarik]

Just adding that the first option might use the magic syntax too, instead of unique names; it would just be restricted to the user's probes.

Also, if we went with unique names, I'd say we make them assigned by us, not user-editable, to have a clear structure, like Atlas's FQDN field. #619 might make lots of sense in that case (possibly still opt-in).

[alexey-yarmosh]

Looks like "apply the 'magic' logic to targets" is the most simple and elegant solution that solves the issue of targeting fully.
Also if we add passing of user tags in env var during probe start, it will solve loosing of target during CI/CD. Tags are private and fully controlled by the user.

A few questions for context:

1. Resolving of e.g. name to the target IP happens on the GP API side? Then resolved target IP is passed to the source probe?
2. Probe is not a server, so required functionality to handle all kinds of measurements will be added to probe?

[MartinKolarik]

Resolving of e.g. name to the target IP happens on the GP API side? Then resolved target IP is passed to the source probe?

Depends on the exact implementation. If we went with DNS, the probe can resolve it, otherwise the API.

Probe is not a server, so required functionality to handle all kinds of measurements will be added to probe?

No, nothing would be added to the probe. Depending on the setup, you'll likely be able to do ping/traceroute/mtr. They might not reach the probe at all if it's behind a router. If you run the docker version on the same server as some web service, HTTP might work too. In all cases, the probe is used for targeting, but it is not necessarily the target.

Looks like "apply the 'magic' logic to targets" is the most simple and elegant solution that solves the issue of targeting fully.

Kind of, but it could also make the targeting quite confusing. I somewhat like it, but the FQDN approach makes sense as well; it doesn't require any magic to work and might even be useful to users in other ways.

[jimaek]

I wasn't clear but yeah, magic would be used in targeting too, in both cases, I just wanted to point out that we need individual probe targeting as well.

And as Martin say, the probe is the target for the request, but not guaranteed to be the processor. So nothing happens on the probe itself, its all on the API

[MartinKolarik]

I wasn't clear but yeah, magic would be used in targeting too, in both cases, I just wanted to point out that we need individual probe targeting as well.

Not necessarily; it's possible for both cases, but it's still just one of the options. The way as see it, we have four options in total covering the discussed behavior:

• users target only their probes, using the existing magic filters,
• users target only their probes but via a globally unique name like FQDN (this one doesn't seem to make much sense),
• users target anyone's probes (but it's still opt-in) using the existing magic filters,
• users target anyone's probes (but it's still opt-in) but via a globally unique name like FQDN.

I definitely think if we do this, it should be the "anyone with opt-in" version, but the different targeting ways each has some pros and cons.

[jimaek]

When you say FQDN, do you mean an actual public domain anyone externally could resolve?
The issue with that then anyone could use our chosen domain to host website and create abuse issues.

So this sounds like the best option

users target anyone's probes (but it's still opt-in) using the existing magic filters,

But with internal unique names to target individual probes

[MartinKolarik]

Yes, I do mean a real domain name, because then it's much more useful and gives people motivation to enable and use it. It could be, I guess as the fifth option, a globally unique name that's only resolvable within GP, but I don't particularly like such limitation.

[jimaek]

Well for security reasons we need to have that limitation I think. I don't want to later spend dev time building anti spam filters for script kiddies that will try hosting phishing sites and malware under our domains.

Although I agree it's more useful to have real FQDN.

[MartinKolarik]

I'm not convinced it's a significant problem, especially if it's a separate domain only for the probes, and we could also add some limitations around it.

[jimaek]

The only limitation I would accept while running the domain under our brand is blocking ports 80/443. But that won't work of course.

Or cross integration with a third party.

[MartinKolarik]

The only limitation I would accept while running the domain under our brand is blocking ports 80/443.

That's something I considered, too, just not sure about the usefulness impact.

[alexey-yarmosh]

Won't FQDN change on every instance replacement via CI/CD?