Skip to content

Conversation

@nseidan
Copy link
Collaborator

@nseidan nseidan commented Nov 5, 2025

  • added some logs (info/debug level)

@nseidan nseidan changed the title DAGE-104: Added logs DAGE-104: Add info/debug level logging Nov 5, 2025
Copy link
Collaborator

@nicolo-rinaldi nicolo-rinaldi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to:

  • change the message logged in the dataset_generator/config.py?
  • log the payloads/url used in the requests?


# score initial docset
if config.enable_cartesian_product:
log.debug("Cartesian product is enabled, so adding cartesian product scores")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would put this inside its function

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

@nseidan
Copy link
Collaborator Author

nseidan commented Nov 5, 2025

change the message logged in the dataset_generator/config.py?

what messages you want to change?

log the payloads/url used in the requests?

I think we should not log the entire payload, because it might be lengthy, if we log at the debug level, we shoudl truncate it.
But we should not log at the info level, because of security reason. Currently we dont support redacted ones. Personally i have never seen payload to be logged, unless to log particular fields.

@nicolo-rinaldi
Copy link
Collaborator

nicolo-rinaldi commented Nov 5, 2025

I would change this into something that says that the config file is the one related to the dataset generator, as we did for the other 2 modules.

For logging the payload, I agree to log it in debug mode (even before, but I didn't specify it). I just think it is very useful when debugging the code to have the payload/url printed, so I can see what the code is actually doing.

@nseidan
Copy link
Collaborator Author

nseidan commented Nov 6, 2025

partial payload is (first 500chars) logged at the debug level

@nseidan nseidan merged commit bc81495 into dataset-generator Nov 6, 2025
3 checks passed
@nseidan nseidan deleted the DAGE-104_logging branch November 6, 2025 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants