Skip to content

Conversation

Pradoxzon
Copy link
Contributor

@Pradoxzon Pradoxzon commented Jul 26, 2023

Adding a feature to allow PaC to create a Job Summary and a PR comment (when running in a PR) that give a summary of the policy violations from the workflow run.
I used the dependency review action as a reference for the implementation.

Job Summary

The job summary appears in the Summary screen of the workflow run. This feature creates this summary with every run, and if the --display flag is present it adds details about the policy violations to the summary. Without the --display flag, the summary only contains details that are present in the log output of the action. This just allows users to get a clean and concise view of the results without digging into the logs.

Enabling PR comments

I added an argvs flag called --pr-comment that enables/disables the PR comment functionality.
This would be replaced with updated flags set in the policy files in v3, see #42 for more details.

The comment added to a PR is exactly the same as the summary added to the Job Summary, and it likewise contains additional details when the --display flag is enabled. PaC will check if it has already added a comment to the PR previously and will update that comment if found. PaC comments are identified by an unique HTML comment in the body of the PR comment.

Summary :: summary.py

The summary.py file defines the Summary static class and its functions for formatting text and adding text to the summary.
Formatting is done by adding HTML elements to the text.

The Summary class also contains a static function, outputJobSummary(), that outputs the summary to the Job Summary. This function uses the GITHUB_STEP_SUMMARY environment variable which is set by GitHub Actions at runtime. The variable points to a temporary file on the runner, and the contents of that file are used as the Job Summary when the job completes. See this doc for reference.

Pull Request :: pullrequest.py

The pullrequest.py file defines the PullRequest static class. The main function for working with PR comments is addPrComment() with findComment() as a helper function.
The addPrComment() function checks if the --pr-comment flag is enabled and also checks if the action is running in a PR. It also automatically checks for existing PaC comments and updates an existing comment if found instead of adding a new comment. The policy name is used as part of the unique identifier for PaC comments. This allows multiple PaC policies that are run on the same repo to maintain separate comments and avoid contention from trying to update the same comment and overwriting each other.
'Unknown' is used when policies do not have a set name. And multiple policies that use the same name or multiple policies that do not have a name will still results in contention over updating the same comment.

Adding content to the Summary

Text is added to the Summary class during the execution of PaC and output to the Job Summary and PR comment (if enabled) at the end.

__main__.py adds:

  • lvl 1 header at the beginning after parsing arguments
  • after the Policy class is initialized, lvl 4 header with the policy name if the policy contains a name
  • before the final results checks, outputs the summary to Job Summary and PR comment as appropriate

checks.py adds for each technology check performed:

  • lvl 2 header for the technology
  • the number of violations or warnings
  • a collapsed table with details about the violations or warnings (only if --display is enabled)

Notes

This implementation requires the changes added to ghastoolkit in PR 78.

@Pradoxzon
Copy link
Contributor Author

I know Black formatting checks are failing with the current state. I'll be able to pull the code locally and fix the formatting issues later today.

@Pradoxzon
Copy link
Contributor Author

@GeekMasher this is a draft for #42.
If it looks good enough, I'd be happy to utilize this feature with my users before v3, but it's not urgent.
Otherwise, this should be able to at least be used as a base for implementing in v3.

@GeekMasher
Copy link
Contributor

ghastoolkit==0.6.2 should help with this PR.

https://github.com/GeekMasher/ghastoolkit/releases/tag/0.6.2

@Pradoxzon
Copy link
Contributor Author

Pradoxzon commented Jul 26, 2023

ghastoolkit==0.6.2 should help with this PR.

https://github.com/GeekMasher/ghastoolkit/releases/tag/0.6.2

@GeekMasher I am unable to run the vendor update.sh script to bump the ghastoolkit version to 0.6.2 with my current setup. Would you be able to run the script to bump the version? If you're able, I can pull the changes.

If not I can give it a shot in a few hours when I have a different setup available.

@Pradoxzon Pradoxzon marked this pull request as ready for review July 26, 2023 19:04
@Pradoxzon Pradoxzon requested a review from GeekMasher as a code owner July 26, 2023 19:04
@GeekMasher GeekMasher mentioned this pull request Jul 26, 2023
@Pradoxzon
Copy link
Contributor Author

Thank you for the help @GeekMasher !

@GeekMasher
Copy link
Contributor

Apart from those comments, we are good to go! Thank you for the PRs 🚀

@GeekMasher GeekMasher mentioned this pull request Jul 26, 2023
@GeekMasher GeekMasher merged commit 37aace5 into advanced-security:main Jul 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants