Skip to content

Streamline option handling #530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Aug 10, 2023
Merged

Streamline option handling #530

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c20b846
Set fuzz target return type to unknown instead of void
bertschneider Jul 21, 2023
16ea262
Add clean task to main npm script
bertschneider Jul 31, 2023
3d9773c
Streamline option handling
bertschneider Jul 31, 2023
51636f2
Switch return value test to new test wrapper
bertschneider Jul 31, 2023
7d92256
Mitigate CLI argument type conversion
bertschneider Aug 1, 2023
11e6ff3
Relax FuzzedDataProvider test
bertschneider Aug 9, 2023
be9ff59
Relax value profiling test
bertschneider Aug 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/fuzz-settings.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,28 @@

This page describes advanced fuzzing settings.

## Configuration options

Jazzer.js can be configured in multiple ways depending on the concrete use case.

The `Options` interface in the [options.ts](../packages/core/options.ts) file
describes all available settings. These can be set via CLI argument, environment
variable or in integration specific ways, e.g. Jest configuration files.

In general the following preference applies with increasing priority:

- Default values from the [`defaultOptions`](../packages/core/options.ts) object
(names in camel case format, e.g. `fuzzTarget`)
- Environment variables (names in upper snake case format with `JAZZER_` prefix,
e.g. `JAZZER_FUZZ_TARGET=Foo`)
- CLI arguments (names in lower snake case format, e.g. `--fuzz_target=Foo`)
- Integration specific configuration (e.g. `jazzerjsrc` or Jest configuration
files)

**Note**: The CLI provides abbreviations for common arguments, e.g. `--includes`
can be abbreviated to `-i`. Only the main argument name is supported in other
configuration approaches, though.

## Corpus

Jazzer.js generates meaningful inputs to a fuzz target based on coverage and
Expand Down
39 changes: 20 additions & 19 deletions docs/fuzz-targets.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,39 +168,40 @@ jazzer <fuzzTarget> <fuzzerFlags> [corpus...] [-- <fuzzingEngineFlags>]
Detailed documentation and some example calls are available using the `--help`
flag, so that only the most important parameters are discussed here.

| Parameter | Description |
| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `<fuzzTarget>` | Import path to the fuzz target module. |
| `[corpus...]` | Paths to the corpus directories. If not given, no initial seeds are used nor interesting inputs saved. |
| `-f`, `--fuzz_function` | Name of the fuzz test entry point. It must be an exported function with a single [Buffer](https://nodejs.org/api/buffer.html) parameter. Default is `fuzz`. |
| `-i`, `--instrumentation_includes` / `-e`, `--instrumentation_excludes` | Part of filepath names to include/exclude in the instrumentation. A tailing `/` should be used to include directories and prevent confusion with filenames. `*` can be used to include all files. Can be specified multiple times. Default will include everything outside the `node_modules` directory. If either of these flags are set the default value for the other is ignored. |
| `--sync` | Enables synchronous fuzzing. **May only be used for entirely synchronous code**. |
| `-h`, `--custom_hooks` | Filenames with custom hooks. Several hooks per file are possible. See further details in [docs/fuzz-settings.md](fuzz-settings.md). |
| `--help` | Detailed help message containing all flags. |
| `-- <fuzzingEngineFlags>` | Parameters after `--` are forwarded to the internal fuzzing engine (`libFuzzer`). Available settings can be found in its [options documentation](https://www.llvm.org/docs/LibFuzzer.html#options). |
| Parameter | Description |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `<fuzzTarget>` | Import path to the fuzz target module. |
| `[corpus...]` | Paths to the corpus directories. If not given, no initial seeds are used nor interesting inputs saved. |
| `-f`, `--fuzz_function` | Name of the fuzz test entry point. It must be an exported function with a single [Buffer](https://nodejs.org/api/buffer.html) parameter. Default is `fuzz`. |
| `-i`, `--includes` / `-e`, `--excludes` | Part of filepath names to include/exclude in the instrumentation. A tailing `/` should be used to include directories and prevent confusion with filenames. `*` can be used to include all files. Can be specified multiple times. Default will include everything outside the `node_modules` directory. If either of these flags are set the default value for the other is ignored. |
| `--sync` | Enables synchronous fuzzing. **May only be used for entirely synchronous code**. |
| `-h`, `--custom_hooks` | Filenames with custom hooks. Several hooks per file are possible. See further details in [docs/fuzz-settings.md](fuzz-settings.md). |
| `--help` | Detailed help message containing all flags. |
| `-- <fuzzingEngineFlags>` | Parameters after `--` are forwarded to the internal fuzzing engine (`libFuzzer`). Available settings can be found in its [options documentation](https://www.llvm.org/docs/LibFuzzer.html#options). |

## Coverage report generation

To generate a coverage report, add the `--cov`/`--coverage` flag to the
Jazzer.js CLI. In the following example, the `--cov` flag is combined with the
dry run flag `-d` that disables internal instrumentation used by the fuzzer.
To generate a coverage report, add the `--coverage` flag to the Jazzer.js CLI.
In the following example, the `--coverage` flag is combined with the mode flag
`-m=regression` that only uses existing corpus entries without performing any
fuzzing.

```shell
npx jazzer -d <fuzzer parameters> --corpus <corpus directories> --cov -- <libFuzzer parameters>
npx jazzer -m=regression <fuzzer parameters> --corpus <corpus directories> --cov -- <libFuzzer parameters>
```

Alternatively, you can add a new script to your `package.json`:

```json
"scripts": {
"coverage": "jazzer -d -i target -i another_target <fuzzer parameters> --corpus <corpus directories> --cov -- <libFuzzer parameters>"
"coverage": "jazzer -m regression -i target -i another_target <fuzzer parameters> --corpus <corpus directories> --cov -- <libFuzzer parameters>"
}
```

Files matched by the flags `--instrumentation_includes` or `--custom_hooks`, and
not matched by the flag `--instrumentation_excludes` will be included in the
coverage report. It is recommended to disable coverage report generation during
fuzzing, because of the substantial overhead that it adds.
Files matched by the flags `--includes` or `--custom_hooks`, and not matched by
the flag `--excludes` will be included in the coverage report. It is recommended
to disable coverage report generation during fuzzing, because of the substantial
overhead that it adds.

### Coverage report directory

Expand Down
2 changes: 1 addition & 1 deletion examples/protobufjs/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"type": "module",
"scripts": {
"fuzz": "npx jazzer fuzz --sync -i protobuf",
"dryRun": "npx jazzer fuzz -d --sync -i protobuf -- -runs=100 -seed=123456789"
"dryRun": "npx jazzer fuzz -m regression --sync -i protobuf -- -runs=100 -seed=123456789"
},
"dependencies": {
"protobufjs": "^7.0.0",
Expand Down
Loading