Skip to content

Conversation

@GioeleB00
Copy link
Collaborator

@GioeleB00 GioeleB00 commented Sep 7, 2025

Summary

This PR makes it much easier to validate AsyncFlow against M/M/1 theory and to analyze scalability by sweeping concurrent users. It adds a pretty, one-call MM1 comparison, a new SweepAnalyzer for plotting results across a user grid, exposes Sweep in the public facade, and ships a guided Jupyter notebook.


What’s new

1) MM1 “theory vs observed” pretty print (built-in)

  • MM1.print_comparison(payload, results)
    Prints a nicely formatted table (λ, μ, ρ, L, Lq, W, Wq) with abs/rel deltas.
    Uses the class’ validation (single server, CPU-exp step, tiny deterministic edges, Poisson arrivals).
  • MM1.compare_and_format(...) returns the string version (for logs/CLI).
  • The builder example was updated to print the MM1 table before rendering plots.

2) User-sweep analysis & plots

  • New module: asyncflow.metrics.sweep_analyzer

    • Global (system) plots vs mean concurrent users:

      • Throughput (mean RPS)
      • Mean latency W (+ optional P50/P95/P99)
    • Per-server overlay plots (works with 1+ servers):

      • Utilization ρᵢ, waiting time Wqᵢ, service rate μᵢ, throughput λᵢ
      • NEW: server latency (waiting+service) mean
    • One-time caching of collected metrics to avoid recomputation across plot calls.

    • Neutral Matplotlib styling (no seaborn, no forced colors), one metric per axes.

  • Public convenience: precollect() warms caches once.

  • Robust selection of “top” servers for overlays (_select_top_servers) based on a chosen metric.

3) Public API

  • asyncflow.__init__ now exports Sweep:

    from asyncflow import AsyncFlow, SimulationRunner, Sweep

4) Guided Jupyter notebook (MM1 end-to-end)

  • Build an M/M/1-friendly scenario (single CPU-exp step, tiny deterministic edges).

  • Explain the arrival sampler (windowed users → piecewise Poisson within windows).

  • Run & compare with MM1.print_comparison(...) (theory vs observed).

  • Plot standard dashboards (latency hist, throughput; server time-series & event metrics).

  • Do a sweep (e.g., 50→250 users, step 5) and render:

    • Global throughput & latency (+ percentiles)
    • Per-server overlays (ρᵢ, Wqᵢ, μᵢ, λᵢ, server latency)

Tests & quality

  • Added unit tests for SweepAnalyzer (global and per-server paths) using lightweight fakes.
  • Tests are Ruff-compliant (naming, import style, assertions) and mypy-clean (explicit types, TYPE_CHECKING blocks, Matplotlib typing via Axes/Figure).
  • Avoid repeated data collection; plotters read from cached results.

Usage snippets

# MM1 comparison (single call)
from asyncflow.analysis import MM1
mm1 = MM1()
mm1.print_comparison(payload, results)  # pretty table in stdout
# Sweep & plots
from asyncflow.runner.sweep import Sweep
from asyncflow.metrics.sweep_analyzer import SweepAnalyzer

pairs = Sweep().sweep_on_user(payload, user_lower_bound=50, user_upper_bound=250, step=5)
sa = SweepAnalyzer(pairs)
sa.precollect()

fig = sa.plot_global_dashboard()               # throughput & mean latency
fig = sa.plot_server_dashboard()               # ρᵢ, Wqᵢ, μᵢ, λᵢ overlays
# Optional: server latency overlay
# sa.plot_server_latency_overlay(ax, server_ids=[...])

Notes / Backwards compatibility

  • No breaking changes to existing APIs.
  • Sweep is now publicly importable from asyncflow (in addition to its original module path).
  • The MM1 comparison is opt-in (one call) and prints only when invoked.

Why this matters

  • Validating queueing-theory fidelity is now a one-liner with an immediately legible table.
  • Teams can sweep load and visualize how throughput and latency evolve, plus per-server hot spots, without writing custom glue.
  • The guided notebook accelerates onboarding and makes the MM1 mapping explicit.

@GioeleB00 GioeleB00 merged commit 722285e into develop Sep 7, 2025
2 checks passed
@GioeleB00 GioeleB00 deleted the feature/multi-processing-event-loop-3 branch September 7, 2025 20:41
@GioeleB00 GioeleB00 changed the title Feature/multi processing event loop 3 PR: MM1 comparison UX + Sweep analytics (global & per-server) + public Sweep + guided notebook Sep 16, 2025
@GioeleB00 GioeleB00 changed the title PR: MM1 comparison UX + Sweep analytics (global & per-server) + public Sweep + guided notebook MM1 comparison UX + Sweep analytics (global & per-server) + public Sweep + guided notebook Sep 16, 2025
GioeleB00 added a commit that referenced this pull request Sep 18, 2025
…q measurement, and metrics plumbing (#29)

* setting up the branch

* minor correction

* minor changes

* improved script for linux

* minor change

* Delete docker_fs/.env.dev

* Delete docker_fs/.env.test

* Features/event generator (#1)

* setting up the generator

* poisson-poisson requests sampling

* Delete docker_fs/.env.dev

* Delete docker_fs/.env.test

* changes

* refactor for event generator

* added gaussian truncated generator and unit tests for helpers

* imprved documentantion and simulation time handling

* simulation start with the vent generator

* small refactor + tests for the generation of events

* minor changes

* Update src/app/schemas/simulation_input.py

Co-authored-by: Copilot <[email protected]>

* Update tests/unit/sampler/test_sampler_helper.py

Co-authored-by: Copilot <[email protected]>

* Update scripts/quality-check.sh

Co-authored-by: Copilot <[email protected]>

* Update tests/integration/db_initialization/test_init_models.py

Co-authored-by: Copilot <[email protected]>

* minor changes after PR review

---------

Co-authored-by: Copilot <[email protected]>

* minor changes

* Features/event generator documentation test improvements (#2)

* moving file to correct folder

* Documentation added

* Update requests_generator.md

* test Added, introduced constants for the sampling window

* Update tests/unit/sampler/test_poisson_posson.py

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* README update

* Improved constants management

* Clean and refactor

* Features/request handler endpoint input (#3)

* Introduction of the input structure for endpoints

* Documentation + foundation of topology input

* Improve Step validation, added unit test for the input

* improved documentation for the input

* Features/definition full payload simulation (#4)

* Improved input structure and pytest

* Improved pytest structure accordingly to the new schema

* definition of the metrics to be measured and update of the simulation input

* improved documentations added rationale behind metrics

* improved pytest logic and code coherence

* Update src/app/core/helpers.py

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* Features/rqs generator runtime (#5)

* definition of state and RqsGeneratorRuntime

* defined edge runtime and more central logic for sampler

* minor changes

* minor changes

* minor bug fixed

* pytest adapted to the new structure, added pytest for rqs_state

* Update src/app/core/runtime/rqs_generator.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/runtime/rqs_generator.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/runtime/edge.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/runtime/edge.py

Co-authored-by: Copilot <[email protected]>

* Update tests/unit/runtime/test_requests_generator.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/event_samplers/common_helpers.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/event_samplers/common_helpers.py

Co-authored-by: Copilot <[email protected]>

* Update src/app/core/event_samplers/common_helpers.py

Co-authored-by: Copilot <[email protected]>

* minor changes

---------

Co-authored-by: Copilot <[email protected]>

* Features/client server runtime (#6)

* defined client runtime + tests

* initiated the server runtime plus correction of tghe structure

* code refactor + test added for runtime + documentation

* Update tests/conftest.py

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

* Features/metric sampler and collection (#7)

* defined architecture for the central collector + documentation

* server modification to collect metrics and updated docs

* Completed metric collector for server plus test

* remove web app dependencies and added metrics for rqs latency

* improved docs and improved metric collection

* changes to make the code compatible with new changes

* improved ci against toml changes

* update lock file

* minor change

* Features/metrics elaboration (#8)

* introduction to the analyzer class + tests

* refactor of the analyzer

* Features/load balancer node (#9)

* Added LB structure in the input for the simulation + tests

* introduced LB runtime + documentation

* Added test for LB and small refactor to define the SimRunner

* Features/simulation runner (#10)

* added the first method to the simulation runner

* added methods to handle nodes

* completed simulation runtime + integration test

* Features/integration tests unit tests (#11)

* new integration test + bug fixing

* added integration and unit test

* first tests working

* minor change

* first working example, upgrade docs, upgrade docstrings

* improvements

* path bug fixed

* new readme and guide to build yaml

* Update README.md

* Added pybuilder and unit tests (#12)

* Refactor/change project name plus docs improvement (#13)

* Changed name from fastsim to asyncflow app folder renamed in asyncflow

* improved docs

* minor changes

* Refactor/pypi preparation (#14)

* Reafctor for the folder schemas + defined import for public api

* Defined public api, improved docs

* minor changes

* minor changes

* Features/lb example and docs tutorial (#15)

* minor docs changes

* improved pytoml

* improved docs + LB examples

* sanity ci check

* Ci for main (#16)

* Readme final, docs improved

* Docs improvement, refactor analyzer, example added

* Added scripts for setup, added pytest, improved readme

* Example LB improvements

* CI for main + system tests

* fixing a bug

* bug fixed

* version bump

* version bump

* Feature/event injection input (#19)

* Structure defined for the pydantic model representing Event to be injected

* Pydantic validation in the payload for events part 1

* Definition of the input for the events injection part 2

* minor fixes ruff and mypy compliant

* full tests and docs added for input event injection

* fixed minor bugs

* Feature/event injection runtime (#20)

* Events injection for edges part 1

* Logic + docs for the event injection to simulate a server down

* Added pydantic validation + unit + int tests for eventinjection

* added exaple for event inj yaml + builder added int tests

* improved server model + system tests for eventinjection

* ruff small fix

* Refactor/fixing readme small fixes (#21)

* new readme, ci to have pycov badge, comment on server runtime

* small changes

* bump version

* Structure input for multiprocessing, improved tests, improved description (#23)

* better test coverage, edges deterministic, step server random (#24)

* new sim value

* Feature/multi processing event loop 3 (#25)

* added metric for server, added logic to collect them + tests

* add kpi to collect in settings default

* added metric plot for server and tests

* Sweep introduction + MM1 heory vs observation + tests

* analysis for sweep results + tests

* mm1 notebook complete + server latency

* Add M/M/c (split) queue-theory analyzer, RANDOM load-balancer policy, and end-to-end comparison reports (#26)

* mmc split + notebook mmc split + random lb algo

* tests for mmc

* New Api RqsGenerator ---> ArrivalsGenerator (#27)

* changing api per ArrivalsGenerator and implementing distrib for ArrivalRuntime

* Reafctoring API Arrival generator, tests integrations now working

* Api Arrival generator migration complete with tests

* Updated plots

* Added edges without network latency plus tests (#28)

* Implemented fcfs with tests

* Implemented fcfs algo and validation of mmc models Erlang C

* small changes

---------

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants