Skip to content

Geoip heuristics setup #338

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
LDiazN merged 8 commits into from
Feb 18, 2026
Merged

Geoip heuristics setup #338

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5799e70
Add clickhouse write url to dev
LDiazN Feb 18, 2026
c27deba
Allow write access to clickhouse in ooniprobe
LDiazN Feb 18, 2026
99ccce2
Set the new schema for the faulty measurements table
LDiazN Feb 18, 2026
2bdf8d3
Change measurement_uid to a unique uid
LDiazN Feb 18, 2026
5619174
Set uid as part of sorting key
LDiazN Feb 18, 2026
fa410fd
use UUID type for uid field
LDiazN Feb 18, 2026
cb4dea0
Add default timestamping to faulty_measurements table
LDiazN Feb 18, 2026
1a5cf5e
point ooniprobe to master
LDiazN Feb 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions scripts/cluster-migration/schema.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,8 +218,9 @@ ALTER TABLE event_detector_changepoints ON CLUSTER oonidata_cluster ADD COLUMN `
-- faulty measurements
CREATE TABLE IF NOT EXISTS faulty_measurements ON CLUSTER oonidata_cluster
(
`ts` DateTime64(3, 'UTC'),
`ts` DateTime64(3, 'UTC') DEFAULT now64(),
`type` String,
`uid` UUID DEFAULT generateUUIDv4(),
-- geoip lookup result for the probe IP
`probe_cc` String,
`probe_asn` UInt32,
Expand All @@ -230,4 +231,4 @@ ENGINE = ReplicatedReplacingMergeTree (
'/clickhouse/{cluster}/tables/ooni/faulty_measurements/{shard}',
'{replica}'
)
ORDER BY (ts, type, probe_cc, probe_asn);
ORDER BY (ts, type, probe_cc, probe_asn, uid);
4 changes: 2 additions & 2 deletions tf/environments/dev/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ data "aws_ssm_parameter" "clickhouse_readonly_test_url" {
}

data "aws_ssm_parameter" "clickhouse_write_url" {
name = "/oonidevops/secrets/clickhouse_write_password"
name = "/oonidevops/secrets/clickhouse_write_url"
}

data "aws_ssm_parameter" "account_id_hashing_key" {
Expand Down Expand Up @@ -554,7 +554,7 @@ module "ooniapi_ooniprobe_deployer" {

service_name = "ooniprobe"
repo = "ooni/backend"
branch_name = "geoip-heuristic"
branch_name = "master"
trigger_path = "ooniapi/services/ooniprobe/**"
buildspec_path = "ooniapi/services/ooniprobe/buildspec.yml"
codestar_connection_arn = aws_codestarconnections_connection.oonidevops.arn
Expand Down
2 changes: 1 addition & 1 deletion tf/environments/prod/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -807,7 +807,7 @@ module "ooniapi_ooniprobe" {
POSTGRESQL_URL = data.aws_ssm_parameter.oonipg_url.arn
JWT_ENCRYPTION_KEY = data.aws_ssm_parameter.jwt_secret.arn
PROMETHEUS_METRICS_PASSWORD = data.aws_ssm_parameter.prometheus_metrics_password.arn
CLICKHOUSE_URL = data.aws_ssm_parameter.clickhouse_readonly_url.arn
CLICKHOUSE_URL = data.aws_ssm_parameter.clickhouse_write_url.arn
ANONC_SECRET_KEY = data.aws_ssm_parameter.anonc_secret_key.arn
}

Expand Down