Skip to content

Implement Dynamic OpenAPI Generation with. OAS 3.0 Support #15729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

softworkz
Copy link
Contributor

@softworkz softworkz commented Aug 6, 2025

Last week, I wanted to generate a .NET ApiClient library for weblate in order to automate a few things.
We're auto-buiilding REST client for a range of languages/platforms, so that they're always up-to-date, so I didn't expect any problems in doing so.

Until I realized that Weblate OpenAPI description are using spec 3.1. That's a problem, because almost nobody is ready for that yet, even some are claiming to be, B
ut when you look closer, you'll realize that an OAD according to OAS 3.1 is largely useless at the time of writing. Most of the tools and tooling hasn't been updated yet, and those who really support 3.1 are entirely new developments - but for 3.1, and they typically do not properly work for 3.0.

That little minor bump is not as minor as it might appear to be. at first sight.

3.1 is an update with a small set of extremely incompatible changes.

It took me several hours and multiple tools to convert the 3.1 from Weblate to a 3.0 version which - most importantly - is actually accepted by 3.0 tools.
Also, the generated document is highly bloated by tons of error response models, spec extensions and inefficiently build component schema. This week, I wanted to try leeting some AI interact with the Weblate API, but those 2MB are way outside of what they are able to ingest.
The latter is the reason for the comprehensive filtering capability: being able to get openapi descriptions creaed in a compact and efficient way.

3.0 doesn't support Webhook descriptions, so I kept 3.1 as the defauilt.
While at it, I also added a Swagger view because ReDoc doesn't have a "Try out" featue.
Finally, the base path was wrong - which had caused all tool and consumers to failing out-of-the-box behavior.
Other changes:

  • Removed the single quote from API doc title
  • The schema generation endpoint (/api/schema/?params...) is included in the API doc itselff now
  • Fix a typo in a drf config function name
  • The OAS version is now included in the title,
    which also determines the file name when downloading
  • Also the title will include 'Light' when light param is set, or "Subset" in all other cases when some filtering params are set

This is hwo it looks with those changes applied:

image

@softworkz softworkz requested a review from nijel as a code owner August 6, 2025 12:05
@softworkz
Copy link
Contributor Author

I've taken the default 3.1 version, the 3.0 version and a 3.0 Light version and ran it through several OpenApui validation tools (swagger-cli, Redocly CLI, Prance and openapi‑spec‑validator or similar)

npx ajv-cli validate --spec=draft2020 -s openapi-3.1.0-schema.json -d '/mnt/c/Users/admin/Downloads/Weblate_3.1_invalid.yaml' --strict=false

All passed - Redocly CLI showed a number of warnings only.

Copy link

codecov bot commented Aug 6, 2025

Codecov Report

❌ Patch coverage is 65.75342% with 75 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.27%. Comparing base (4c4be11) to head (930cd54).

Files with missing lines Patch % Lines
weblate/api/spectacular.py 63.68% 45 Missing and 28 partials ⚠️
weblate/api/spectacular_settings.py 87.50% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main   #15729      +/-   ##
==========================================
- Coverage   91.36%   91.27%   -0.09%     
==========================================
  Files         645      646       +1     
  Lines       67307    67510     +203     
  Branches     6930     6992      +62     
==========================================
+ Hits        61492    61623     +131     
- Misses       4068     4112      +44     
- Partials     1747     1775      +28     
Files with missing lines Coverage Δ
weblate/api/urls.py 100.00% <100.00%> (ø)
weblate/middleware.py 82.07% <ø> (ø)
weblate/urls.py 82.27% <100.00%> (ø)
weblate/api/spectacular_settings.py 87.50% <87.50%> (ø)
weblate/api/spectacular.py 63.86% <63.68%> (-23.64%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nijel
Copy link
Member

nijel commented Aug 6, 2025

Spectacular should be able to create a 3.0 schema directly without needing to maintain the conversion code. Unfortunately, it seems to be settings only (OAS_VERSION). Have you considered changing the configuration only instead of writing the conversion code?

@softworkz
Copy link
Contributor Author

softworkz commented Aug 6, 2025

There is no conversion code. I'´m setting OAS_VERSION to 3.0 and spectacular creates a 3.0 description.

The code that is there has different purposes:

_filter_patterns()

That's the provided way by SpectacularAPIView for filtering out certain paths. I'm pretending to do this "by Tags", while in reality, paths are filtered by the first path segment. But as this is always the same as the tag, it's essentially that: "filter by tag"

_prune_unused_tags

While the above is filtering out the paths, it doesn't filter out the actual Tag elements. _prune_unused_tags() does this and so it avoids that there are lots of empty sections in the left column when certain paths are filtered out.

_prune_unused_components()

The next problem is that when we filter by using the patterns parameter, sperctacular omits the compoinent schemas from the directly related definitions, but it still leaves most of it in the document.
_prune_unused_components() recursively parses the components and follows all references in order to detect and remove everything that is not referenced.
Validations have shown that it is working pretty well like this (even though I would had preferred not needing to do this manually).

_collect_used_security_schemes()

The only purpose of this function is to identify the security_scheme items, because they would otherwise be cleaned up by _prune_unused_components().

_strip_examples()

Does what it says for data examples.

Note: The code for stripping out the errors (when chosen) is still inline, it should probably be moved out for consistency.

Both of the latter play together with _prune_unused_components(). They are removing the directly related elements and _prune_unused_components() does the deep/recursive cleanup.

@softworkz
Copy link
Contributor Author

This means in turn:

For a URL like .../api/schema(?oav=3.0, none of this code would need to run. At the moment it does run, but the only actual effect comes from _prune_unused_tags(), which means that those tags get remvoed which do not have any content either way (i.e. Glossary and Tasks).

@nijel
Copy link
Member

nijel commented Aug 7, 2025

Thanks for the explanation. I really didn't look at the patch as it's really hard to review because of moving code to another file and then adding new code instead of it - the diff is really messy. Having the fixes separate from the new feature would make it much easier to review. I will get to reviewing the patch, but probably not before the 5.13 release. If you can split it up, the fixes will be easier to review and can be processed sooner.

@softworkz
Copy link
Contributor Author

If you can split it up, the fixes will be easier to review and can be processed sooner.

Have you seen that there are 9 separate commits, each one doing a very specific change only?

@softworkz
Copy link
Contributor Author

softworkz commented Aug 7, 2025

The second commit (a2c3e57) moves the configuration without making an actual change to the contents, so that content changes can be reviewed properly.

(you just need to review commit-by-commit)

@softworkz
Copy link
Contributor Author

If you prefer the drf settings code to remain in the spectacular.py file, I can reshape the PR.
I could also move out the helper code into a separate py file (like openapi_utils.py or similar.

The original reason for moving it was that I had encountered once another circular referencing problem, but I think it resolved naturally, later. I still found it reasonable to keep it separate, though, but there are many valid way for structuring things and I'm good with whichever way you like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants