Skip to content

Commit 806cf71

Browse files
prabhuprabhu
authored
Support for app-only vdb (#360)
* Support for app-only vdb Signed-off-by: Prabhu Subramanian <[email protected]> * Update readme Signed-off-by: Prabhu Subramanian <[email protected]> --------- Signed-off-by: Prabhu Subramanian <[email protected]>
1 parent 1088d87 commit 806cf71

File tree

6 files changed

+883
-700
lines changed

6 files changed

+883
-700
lines changed

‎Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,14 +89,14 @@ RUN set -e; \
8989
&& echo 'extension=timezonedb.so' >> /etc/php.ini \
9090
&& php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && php composer-setup.php \
9191
&& mv composer.phar /usr/local/bin/composer \
92-
&& python3 -m pip install pipenv certifi \
92+
&& python3 -m pip install --no-cache-dir --upgrade pipenv certifi \
9393
&& curl -LO https://github.com/dragonflyoss/nydus/releases/download/v${NYDUS_VERSION}/nydus-static-v${NYDUS_VERSION}-linux-${GOBIN_VERSION}.tgz \
9494
&& tar -xvf nydus-static-v${NYDUS_VERSION}-linux-${GOBIN_VERSION}.tgz \
9595
&& chmod +x nydus-static/* \
9696
&& mv nydus-static/* /usr/local/bin/ \
9797
&& rm -rf nydus-static-v${NYDUS_VERSION}-linux-${GOBIN_VERSION}.tgz nydus-static \
9898
&& cd /opt/dep-scan \
99-
&& python3 -m pip install -e . \
99+
&& python3 -m pip install --no-cache-dir -e . \
100100
&& chmod a-w -R /opt \
101101
&& rm -rf /var/cache/yum \
102102
&& microdnf clean all

‎README.md

Lines changed: 54 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@ OWASP dep-scan is a next-generation security and risk audit tool based on known
1414
- [Linux distros](#linux-distros)
1515
- [Usage](#usage)
1616
- [OCI Artifacts via ORAS cli](#oci-artifacts-via-oras-cli)
17-
- [Single binary executables](#single-binary-executables)
1817
- [Server mode](#server-mode)
1918
- [Scanning projects locally (Python version)](#scanning-projects-locally-python-version)
2019
- [Scanning containers locally (Python version)](#scanning-containers-locally-python-version)
@@ -278,25 +277,35 @@ dep-scan uses [cdxgen](https://github.com/CycloneDX/cdxgen) command internally t
278277
279278
The following projects and package-dependency format is supported by cdxgen.
280279
281-
| Language | Package format |
282-
| ------------------------ | --------------------------------------------------------------------------------------- |
283-
| node.js | package-lock.json, pnpm-lock.yaml, yarn.lock, rush.js, bower.json, .min.js |
284-
| java | maven (pom.xml [1]), gradle (build.gradle, .kts), scala (sbt), bazel |
285-
| php | composer.lock |
286-
| python | setup.py, requirements.txt [2], Pipfile.lock, poetry.lock, bdist_wheel, .whl, .egg-info |
287-
| go | binary, go.mod, go.sum, Gopkg.lock |
288-
| ruby | Gemfile.lock, gemspec |
289-
| rust | binary, Cargo.toml, Cargo.lock |
290-
| .Net | .csproj, packages.config, project.assets.json [3], packages.lock.json, .nupkg |
291-
| dart | pubspec.lock, pubspec.yaml |
292-
| haskell | cabal.project.freeze |
293-
| elixir | mix.lock |
294-
| c/c++ | conan.lock, conanfile.txt |
295-
| clojure | Clojure CLI (deps.edn), Leiningen (project.clj) |
296-
| docker / oci image | All supported languages and Linux OS packages |
297-
| GitHub Actions Workflows | .github/workflows/\*.yml |
298-
| Jenkins Plugins | .hpi files |
299-
| YAML manifests | docker-compose, kubernetes, kustomization, skaffold, tekton etc |
280+
| Language/Platform | Project Types | Package Formats | Supported Evidence | Supports Transitives |
281+
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------- |
282+
| Node.js | `npm`, `pnpm`, `nodejs`, `js`, `javascript`, `typescript`, `ts`, `tsx` | `npm-shrinkwrap.json`, `package-lock.json`, `pnpm-lock.yaml`, `yarn.lock`, `rush.js`, `bower.json`, `.min.js` | Yes, except for `.min.` | ✅ |
283+
| Java | `java`, `groovy`, `kotlin`, `scala`, `jvm`, `gradle`, `mvn`, `maven`, `sbt` | `pom.xml` [1], `build.gradle`, `.kts`, `sbt`, `bazel` | Yes, unless `pom.xml` is manually parsed due to unavailability of maven or errors) | ✅ |
284+
| Android | `android`, `apk`, `aab` | `apk`, `aab` | - | - |
285+
| JAR | `jar` | `.jar` | - | - |
286+
| JAR (Gradle Cache) | `gradle-index`, `gradle-cache` | `$HOME/caches/modules-2/files-2.1/\*\*/\*.jar` | - | - |
287+
| JAR (SBT Cache) | `sbt-index`, `sbt-cache` | `$HOME/.ivy2/cache/\*\*/\*.jar ` | - | - |
288+
| JAR (Maven Cache) | `maven-index`, `maven-cache`, `maven-repo` | `$HOME/.m2/repository/\*\*/\*.jar` | - | - |
289+
| Python | `python`, `py`, `pypi` | `pyproject.toml`, `setup.py`, `requirements.txt` [2], `Pipfile.lock`, `poetry.lock`, `pdm.lock`, `bdist_wheel`, `.whl`, `.egg-info` | Yes using the automatic pip install/freeze. When disabled, only with `Pipfile.lock` and `poetry.lock` | ✅ |
290+
| Golang | `go`, `golang` | `binary`, `go.mod`, `go.sum`, `Gopkg.lock` | Yes except binary | ✅ |
291+
| Rust | `rust`, `rust-lang`, `cargo` | `binary`, `Cargo.toml`, `Cargo.lock` | Only for `Cargo.lock` | - |
292+
| Ruby | `ruby`, `gems` | `Gemfile.lock`, `gemspec` | Only for `Gemfile.lock` | - |
293+
| .NET (#C) | `csharp`, `netcore`, `dotnet`, `vb`, `dotnet-framework` | `.csproj`, `.vbproj`, `.fsproj`, `packages.config`, `project.assets.json` [3], `packages.lock.json`, `.nupkg`, `paket.lock`, `binary` | Only for `project.assets.json`, `packages.lock.json`, `paket.lock` | - |
294+
| Dart | `dart`, `flutter`, `pub` | `pubspec.lock`, `pubspec.yaml` | Only for `pubspec.lock` | - |
295+
| Haskell | `haskell`, `hackage`, `cabal` | `cabal.project.freeze` | Yes | |
296+
| Elixir | `elixir`, `hex`, `mix` | `mix.lock` | Yes | - |
297+
| C++ | `c`, `cpp`, `c++`, `conan` | `conan.lock`, `conanfile.txt`, `\*.cmake`, `CMakeLists.txt`, `meson.build`, codebase without package managers! | Yes only for `conan.lock`. Best effort basis for `cmake` without version numbers. | ✅ |
298+
| Clojure | `clojure`, `edn`, `clj`, `leiningen` | `deps.edn`, `project.clj` | Yes unless the files are parsed manually due to lack of clojure cli or leiningen command | - |
299+
| GitHub Actions | `github`, `actions` | `.github/workflows/\*.yml` | n/a | ✅ |
300+
| Operation System (OS) | `os`, `osquery`, `windows`, `linux`, `mac`, `macos`, `darwin` |
301+
| Jenkins Plugins | `jenkins` | `.hpi files` | - | ✅ |
302+
| Helm | `helm`, `charts` | `.yaml` | n/a | |
303+
| Helm (Cache) | `helm-index`, `helm-repo` | `$HOME/.cache/helm/repository/\*\*/\*.yaml` | - | - |
304+
| Container | `universal`, `containerfile`, `docker-compose`, `dockerfile`, `swarm`, `tekton`, `kustomize`, `operator`, `skaffold`, `kubernetes`, `openshift`, `yaml-manifest` | `.yaml`, `docker-compose\*.yml`, `*Dockerfile*`, `*Containerfile*`, `bitbucket-pipelines.yml` | n/a | - |
305+
| Google Cloud Build | `cloudbuild` | `cloudbuild.yaml` | n/a | - |
306+
| Swift (iOS) | `swift` | `Package.resolved`, `Package.swift` (swiftpm) | Yes | - |
307+
| Binary | `binary`, `blint` |
308+
| Open API | Open API Specification, Swagger | `openapi\*.json`, `openapi\*.yaml` | n/a | - |
300309
301310
## Reachability analysis
302311
@@ -339,6 +348,30 @@ The following environment variables can be used to customize the behavior.
339348
- VDB_HOME - Directory to use for caching database. For docker-based execution, this directory should get mounted as a volume from the host
340349
- VDB_DATABASE_URL - Vulnerability DB URL. Defaults to: ghcr.io/appthreat/vdbgz:v5
341350
- USE_VDB_10Y - Set to true to use the larger 10-year vulnerability database. Default download url: ghcr.io/appthreat/vdb-10y:v5
351+
- VDB_APP_ONLY - Set to true to use a special app-only vulnerability database. Default download url: ghcr.io/appthreat/vdbgz-app:v5
352+
353+
Example 1 - Run depscan with app-only vdb.
354+
355+
```shell
356+
docker run --rm \
357+
-e VDB_HOME=/db \
358+
-e VDB_APP_ONLY=true \
359+
-e SCAN_DEBUG_MODE=debug \
360+
-v /tmp:/db \
361+
-v $PWD:/app ghcr.io/owasp-dep-scan/dep-scan --src /app --reports-dir /app/reports
362+
```
363+
364+
Example 2 - Run depscan with a larger 10 year app-only vdb.
365+
366+
```shell
367+
docker run --rm \
368+
-e VDB_HOME=/db \
369+
-e VDB_APP_ONLY=true \
370+
-e USE_VDB_10Y=true \
371+
-e SCAN_DEBUG_MODE=debug \
372+
-v /tmp:/db \
373+
-v $PWD:/app ghcr.io/owasp-dep-scan/dep-scan --src /app --reports-dir /app/reports
374+
```
342375
343376
## GitHub Security Advisory
344377
@@ -480,7 +513,7 @@ The objects available are taken from the CycloneDX \*.vdr.json BOM file generate
480513
`pkg_group_rows` - List of vulnerability id and the dependency tree prioritized by depscan.
481514
482515
Furthermore, insights are imaginable to be made available to the template, please reach out or contribute on demand.
483-
We appreciate it if you like to contribute your report templates as examples, please add/find them [here](contrib/report-templates/).
516+
We appreciate it if you like to contribute your report templates as examples, please add/find them [here](contrib/report-templates).
484517
485518
## Performance tuning
486519

‎depscan/lib/config.py

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -314,17 +314,21 @@ def get_int_from_env(name, default):
314314
"VDB_RAFS_DATABASE_URL", "ghcr.io/appthreat/vdb:v5-rafs"
315315
)
316316

317-
# Larger 10 year database
318-
vdb_10y_database_url = os.getenv(
319-
"VDB_10Y_DATABASE_URL", "ghcr.io/appthreat/vdbgz-10y:v5"
320-
)
321-
vdb_10y_rafs_database_url = os.getenv(
322-
"VDB_10Y_RAFS_DATABASE_URL", "ghcr.io/appthreat/vdb-10y:v5-rafs"
323-
)
317+
# App only data
318+
if os.getenv("VDB_APP_ONLY", "") in ("true", "1"):
319+
vdb_database_url = os.getenv("VDB_APP_DATABASE_URL", "ghcr.io/appthreat/vdbgz-app:v5")
320+
vdb_rafs_database_url = os.getenv("VDB_APP_RAFS_DATABASE_URL", "ghcr.io/appthreat/vdb-app:v5-rafs")
324321

325-
if os.getenv("USE_VDB_10Y", "") in ("true", "1"):
326-
vdb_database_url = vdb_10y_database_url
327-
vdb_rafs_database_url = vdb_10y_rafs_database_url
322+
# Larger 10 year database
323+
if os.getenv("USE_VDB_10Y", "") in ("true", "1") or os.getenv("NVD_START_YEAR", "") in ("2014",):
324+
if os.getenv("VDB_APP_ONLY", "") in ("true", "1"):
325+
# 10 year app-only database
326+
vdb_database_url = os.getenv("VDB_APP_10Y_DATABASE_URL", "ghcr.io/appthreat/vdbgz-app-10y:v5")
327+
vdb_rafs_database_url = os.getenv("VDB_APP_10Y_RAFS_DATABASE_URL", "ghcr.io/appthreat/vdb-app-10y:v5-rafs")
328+
else:
329+
# 10 year database
330+
vdb_database_url = os.getenv("VDB_10Y_DATABASE_URL", "ghcr.io/appthreat/vdbgz-10y:v5")
331+
vdb_rafs_database_url = os.getenv("VDB_10Y_RAFS_DATABASE_URL", "ghcr.io/appthreat/vdb-10y:v5-rafs")
328332

329333
# Package risk scoring using a simple weighted formula with no backing
330334
# research All parameters and their max value and weight can be overridden

0 commit comments

Comments
 (0)