Skip to content

Commit d243b53

Browse files
committed
Add code coverage reports
- Add integration tests for source code coverage report generation - Refine the interplay between the flags "dry run" (-d) and "custom hooks" (-h). Now the custom hooking works even in dry run mode. - Simplify the FuzzedDataProvider test to decrease the chances of the test failing on MacOS (this fuzz test is expected to find an exception within some time)
1 parent 7d45887 commit d243b53

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

53 files changed

+1388
-32
lines changed

.config/test-jazzerjsrc.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
22
"includes": ["target"],
33
"excludes": ["nothing"],
4-
"fuzzerOptions": ["-runs=-1", "-rss_limit_mb=16000"]
4+
"fuzzerOptions": ["-rss_limit_mb=16000"]
55
}

docs/fuzz-targets.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,3 +172,39 @@ flag, so that only the most important parameters are discussed here.
172172
| `--sync` | Enables synchronous fuzzing. **May only be used for entirely synchronous code**. |
173173
| `-h`, `--custom_hooks` | Filenames with custom hooks. Several hooks per file are possible. See further details in [docs/fuzz-settings.md](docs/fuzz-settings.md). |
174174
| `--help` | Detailed help message containing all flags. |
175+
176+
## Coverage report generation
177+
178+
To generate a coverage report, add the `--coverage` flag to the Jazzer.js CLI.
179+
In this example, the `--coverage` flag is combined with the dry run flag `-d`
180+
that disables internal instrumentation used by the fuzzer.
181+
182+
```shell
183+
npx jazzer -d <fuzzer parameters> --corpus <corpus directories> --coverage -- <libFuzzer parameters>
184+
```
185+
186+
Alternatively, you can add a new script to your package.json:
187+
188+
```json
189+
"scripts": {
190+
"coverage": "jazzer -d -i target -i another_target -e nothing <fuzzer parameters> --corpus <corpus directories> --coverage -- <libFuzzer parameters>"
191+
}
192+
```
193+
194+
Files matched by the flags `--include` or `--custom_hooks`, and not matched by
195+
the flag `--exclude` will be included in the coverage report. It is recommended
196+
to disable coverage report generation during fuzzing, because of a substantial
197+
overhead that it adds.
198+
199+
### Coverage report directory
200+
201+
By default, the coverage reports can be found in the `./coverage` directory.
202+
This default directory can be changed by setting the flag
203+
`--coverageDirectory=<another coverage directory>`.
204+
205+
### Coverage reporters
206+
207+
The desired report format can be set by the flag `--coverageReports`, which by
208+
default is set to `--coverageReports clover json lcov text`. See
209+
[here](https://github.com/istanbuljs/istanbuljs/tree/master/packages/istanbul-reports/lib)
210+
for a list of supported coverage reporters.

docs/jest-integration.md

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,8 @@ runner.
4747
"name": "jest-integration-example",
4848
"scripts": {
4949
"test": "jest",
50-
"fuzz": "JAZZER_FUZZ=1 jest"
50+
"fuzz": "JAZZER_FUZZ=1 jest",
51+
"coverage": "jest --coverage"
5152
},
5253
"devDependencies": {
5354
"@jazzer.js/jest-runner": "1.1.0",
@@ -283,6 +284,22 @@ Time: 0.335 s, estimated 1 s
283284
Ran all test suites.
284285
```
285286
287+
### Coverage report generation
288+
289+
To generate a coverage report, run jest with the `--coverage` flag:
290+
291+
```shell
292+
npx jest --coverage
293+
```
294+
295+
Additional options for coverage report generation are described in the
296+
[fuzz targets documentation](./fuzz-targets.md#coverage-report-generation).
297+
298+
The desired report format can be set by the flag `--coverageReports`, which by
299+
default is set to `--coverageReports clover json lcov text`. See
300+
[here](https://github.com/istanbuljs/istanbuljs/tree/master/packages/istanbul-reports/lib)
301+
for a list of supported coverage reporters.
302+
286303
## IDE Integration
287304
288305
As the Jest test framework foundations are used by the Jazzer.js fuzz test
@@ -309,6 +326,5 @@ offer good extension points and common test framework features have to be
309326
reimplemented.
310327
311328
- Mock functions
312-
- Coverage generation
313329
- Isolated workers
314330
- Test-based timeouts (third parameter to `test` functions)
44 Bytes
Binary file not shown.
16 Bytes
Binary file not shown.
34 Bytes
Binary file not shown.
77 Bytes
Binary file not shown.
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
��������
40 Bytes
Binary file not shown.
52 Bytes
Binary file not shown.

0 commit comments

Comments
 (0)