OWASP DependencyCheck returns 403 Forbidden accessing NVD API using API key
Introduction
Recently, the NVD (National Vulnerability Database) which the Owasp dependency check plugin uses to get its data from to check for vulnerabilities, has introduced the use of an API key. That's for them to better control access and throttling - imagine how many companies and organizations are using that API, each time a dependency check build is performed. Especially those that don't cache the NVD database and at each run retrieve it again. And be aware: "... previous versions of dependency-check utilize the NVD data feeds which will be deprecated on Dec 15th, 2023. Versions earlier then 9.0.0 are no longer supported and could fail to work after Dec 15th, 2023."
But this introduction doesn't go without some hiccups. For example it is possible to still get HTTP 403 Forbidden responses, even while you have a valid key. Here's my research while trying to fix it.
Setup:
- Gradle 7.x
- Dependency Check v9.0.6 (issue applies at least for versions > 8.4.3)
- Configuration:
dependencyCheck {
failBuildOnCVSS = 6
failOnError = true
suppressionFile = '/bamboo/owasp/suppressions.xml'
nvd.apiKey = '<yourkey>'
}
You can also set it dynamically via an environment variable like this:
dependencyCheck {
nvd {
apiKey = System.getProperty("ENV_NVD_API_KEY")
}
} - Via commandline you can invoke it like this:
./gradlew dependencyCheckAggregate -DENV_NVD_API_KEY=<yourkey>
Solution
First you should check if your API key is valid by execution this command:
curl -H "Accept: application/json" -H "apiKey: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" -v https://services.nvd.nist.gov/rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\*
(where xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is your NVD API key)
That should return JSON (and not a 404). Now you know your API key is valid.
Some have some success with setting the delay longer:
nvd {
apiKey = System.getProperty("ENV_NVD_API_KEY")
delay = 6000 // milliseconds, default is 2000 with API key, 8000 without
}
Commandline version:
--nvdDelay 6000
You can also increase the validForHours option, but that doesn't work if during you construct completely new Docker containers each build - you lose that history.
All NVD options you can pass to DependencyCheck are found here.
But currently (27 December 2023) all the above efforts don't always fix the problem of the 403. Sometimes it works for a while, but then not again. If you build many projects in your company at about the same time, you still have a chance of getting throttled of course.
The best solution is to create a local cache so you are less dependent on NVID API calls (and thus the throttling).
Other causes mentioned
- Being behind a proxy with your whole company, see https://github.com/jeremylong/DependencyCheck/issues/6127 and "If you have multiple builds happening at the same time - using the same API key you could hit the NVD rate limiting threshold. Ideally, in an environment with multiple builds you would implement some sort of caching strategy". See: https://github.com/jeremylong/DependencyCheck/issues/6195
- Use --data argument to control cache location.
- It appears the NVD has put a temporary block on all requests that use a virtualMatchString of "cpe:2.3:a" - outstanding issue.
Misc
- Check the best practices from NVD themselves
Add to:
Del.icio.us
DZone
Reddit
Digg
