Skip to content

I1265 add per problem per minute throttling strategy#1267

Open
clevengr wants to merge 3 commits into
pc2ccs:developfrom
clevengr:i1265_add_per_problem_per_minute_throttling_strategy
Open

I1265 add per problem per minute throttling strategy#1267
clevengr wants to merge 3 commits into
pc2ccs:developfrom
clevengr:i1265_add_per_problem_per_minute_throttling_strategy

Conversation

@clevengr
Copy link
Copy Markdown
Contributor

Description of what the PR does

  • Implements a new Throttling Strategy MaxSubmissionsPerMinutePerProblem which supports submission threshholds on a Per-Problem basis.
  • Makes the new Strategy the default PC2 strategy.

Issue which the PR addresses

Fixes #1265

Environment in which the PR was developed (OS,IDE, Java version, etc.)

Windows 11, java version "1.8.0_381"

Precise steps for testing the PR:

  1. Download and unzip the PR distribution.
  2. Start a PC2 Server using --load sumithello.
  3. Start a PC2 Admin client and login as administrator1.
  4. Create a new "FEEDER" account with the Shadow Proxy Team permission. This is done as follows:
    • On the Configure Contest -> Accounts tab, select the Generate button.
    • Generate 1 FEEDER account (select FEEDER from the combo-box, and enter a 1 in the box to the right of it.)
    • Press the Generate Accounts for Site 1 button.
    • Close the Generate Account dialog
    • Select the newly added "feeder1" account and press the Edit button.
    • In the Permissions/Abilities pane on the right, check the Shadow Proxy Team check box, and press Update.
  5. Enable CCS Test Mode. This is done as follows:
    • On the administrator Configure Contest -> Settings tab, scroll all the way to the bottom, and check the Enable CCS Test Mode in the CCS Test Mode group box, then press the Update button.
  6. Start an event feeder client and log in using the newly created "feeder1" account.
  7. On the Web Services tab, press the Start button to start the CLICS event feeder.
  8. On the PC2 Admin, use the Times tab to start the contest running.
  9. Switch to the Run Contest -> Runs tab on the administrator client.
  10. From a command prompt, navigate to the folder where you installed PC2 (or to the main pc2v9 source folder if you're running PC2 from an IDE).
  11. Run: python bin/pc2submit -u https://localhost:50443 -t 1 -p a -y samps/src/hello.cpp.
    • Note that this requires Python on your machine.
    • This makes a submission from Team 1 (-t 1) for Problem A (-p a).
  12. Verify the submission appears on the PC2 Admin's Runs tab.
  13. Re-execute the submission command 7 more times in quick succession (<1 minute). After the 6th submission verify that you get an error message on the console like Submission threshold exceeded - wait a minute and try again.
  14. Edit the pc2v9.ini file and add (or change) the lines toward the end of the file as shown:
    [throttle-strategy-settings]
    maxSubmissionsPerMinutePerProblem=1
    
    This will limit the system to one submission PER PROBLEM per minute (be sure to note carefully the full spelling in the second line!)
  15. Close and restart the "feeder1" event feed client. This will cause it to re-read the pc2v9.ini file to use the new throttle settings.
  16. Press the Start button on the event feeder client to start the CLICS API.
  17. Now try to make 2 submissions (as described above) in quick succession. Since we set the limit to 1 per minute, only the first one should succeed, the second should get an error indicating Submission threshold exceeded - wait a minute and try again.
  18. Quickly re-execute the submission command, but change the -p a to -p b. This will make a submission in the same minute BUT FOR A DIFFERENT PROBLEM (B).
  19. Verify that the submission for problem B is accepted even though the limit for problem A has been reached.
  20. Open a command prompt (terminal) in the PC2 Distribution's "projects" folder and unzip the WebTeamInterface.zip folder.
  21. Change to the unzipped WTI distribution folder and start the WTI Server, using the command ./bin/pc2wti.
  22. Open a browser and login to the WTI using a URL like http://localhost:8080 and login credentials "team1/team1".
  23. Watch the system time clock on your machine and wait until it clicks over to a new wall-time minute, then use the WTI GUI to quickly submit SEVEN submissions in a row for the same problem. (It's challenging but it's doable...).
  24. Verify that the 7th submission is rejected with a red message like Submission threshhold exceeded.
    (If you don't get the rejection message on the 7th submission, look at the WTI Runs tab and verify that all seven have the SAME "Time" value. If not, you didn't get all the submissions in during the same wall-time minute and therefore you might not have gotten them all within the same 60-second threshhold.)
  25. Log out of the WTI and close the WTI browser window.
  26. Kill the WTI Server.
  27. Edit the pc2v9.ini file IN THE WTI DISTRIBUTION FOLDER by adding the following to the bottom:
    [throttle-strategy-settings]
    maxSubmissionsPerMinutePerProblem=1
    
  28. Restart the WTI Server.
  29. Open a new browser window and login to the WTI client.
  30. Quickly submit two runs for the same problem one after another.
  31. Verify that the second run is rejected (because the threshhold is now 1 per problem per minute).
  32. Quickly submit another run for a DIFFERENT problem.
  33. Verify that the new submission is accepted. (If you submitted the run for a different problem in the same minute as the first two submission, this verifies that the WTI enforces threshhold limits PER PROBLEM.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request In PR code review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a "_per problem_ per minute" throttling strategy

1 participant