Keep third-party licenses in oslili aggregation without attributing them to the package - #56
Merged
Merged
Conversation
osslili 1.7.0 categorizes licenses found in bundled third-party notice files (THIRD_PARTY_NOTICES.txt, 3rdpartylicenses.txt) as "third-party" instead of "declared". The aggregation hard-coded the three legacy categories, so those licenses were silently dropped from the license list and from primary-license selection. Capture a third_party bucket and append it last. Third-party licenses are recorded again, but rank behind declared/detected/referenced so a bundled dependency license can never be picked as the project's own license via all_licenses[0]. No-ops against pre-1.7.0 osslili, where no license carries that category, so there is no release-ordering constraint. Closes #55
Recording third-party licenses in the aggregation made them reachable by the primary-license selection in enhance_package_match(): when a scan found nothing but bundled notice files, all_licenses[0] was a third-party license and became match.license. Previously those licenses were dropped entirely, so the list was empty and match.license was left untouched. Attributing a bundled dependency's license to the package is exactly the misattribution the new osslili category exists to prevent, and a high-confidence notice file could even overwrite an already-known license. detect_licenses() now also returns a split view, own_licenses and third_party_licenses, mirroring the third_party_licenses field osslili added to its KissBOM output. enhance_package_match() picks the primary from own_licenses only and leaves match.license untouched when a scan turned up nothing but bundled notices. Third-party licenses are still reported, under metadata["third_party_licenses"]. The new keys are additive and present on every return path, so existing callers reading "licenses" are unaffected.
Two more spots where recording third-party licenses let them reach a project-license decision: package_identifier.py joined the first three entries of "licenses" into match.license, so a scan that only found bundled notice files named a dependency's license as the package's. It now uses own_licenses and leaves the field empty when there is no own license. The primary-license gate in enhance_package_match() compared the aggregate confidence, which now averages own and third-party detections together. Enough high-confidence notice files could push that average past the 0.85 overwrite threshold and replace an already-known license on the strength of a weak own detection. detect_licenses() therefore also reports own_confidence, covering own licenses only, and the gate uses it. For a scan with no third-party licenses the two scores are identical, so existing behaviour is unchanged. The CLI's local-source report still lists every detected license: it describes what was found rather than attributing a license to the package, which is the distinction that matters here.
The outer gate in enhance_package_match() still compared the aggregate confidence, which averages own and third-party detections together. A pile of low-confidence bundled notice files drags that average under the 0.7 threshold, so the whole block was skipped and a high-confidence own license was never applied: one declared license at 0.95 plus four notice files at 0.1 scores 0.27 overall and left match.license unset. This is the mirror image of the inflation case already fixed. Bundled notices should neither raise nor lower the bar for the project's own license, so the gate now also passes when own_confidence clears it, and own_licenses/own_confidence are resolved before the gate. With no third-party licenses present the two scores are equal, so behaviour is unchanged.
License Check Reportℹ️ No licenses detected in changed files. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #55.
Problem
osslili 1.7.0 (SemClone/osslili#87) added a
THIRD_PARTYlicense category. Licenses found in bundled third-party notice files (THIRD_PARTY_NOTICES.txt,3rdpartylicenses.txt, ...) are now categorizedthird-partyinstead ofdeclared.src2id/integrations/oslili.pyhard-coded the three legacy categories when aggregating, so those licenses were silently dropped from the license list and from primary-license selection.Changes
The first commit is the fix from the issue: capture a
third_partybucket and append it last, so third-party licenses are recorded but rank behind declared/detected/referenced.Recording them again, however, made them reachable by the code that decides a package's own license, which could not happen while they were being dropped. The remaining commits close the three paths where a bundled dependency's license could have been attributed to the package itself:
enhance_package_match()assignedall_licenses[0]tomatch.license, so a scan that found nothing but notice files named a dependency's license as the package's.package_identifier.pyjoined the first three entries oflicensesintomatch.license, with the same effect.detect_licenses()therefore also reports a split view,own_licenses/third_party_licenses/own_confidence, mirroring thethird_party_licensesfield osslili added to its KissBOM output. Primary-license decisions use the own values; third-party licenses are still reported, undermetadata["third_party_licenses"].The new keys are additive and present on every return path, so existing callers reading
licensesorconfidenceare unaffected. When a scan finds no third-party licenses,own_confidenceequalsconfidenceand behaviour is identical to before.The CLI's local-source report still lists every detected license. It describes what was found rather than attributing a license to the package, which is the distinction driving the changes above.
Backward compatibility
Nothing carries the
third-partycategory before osslili 1.7.0, so this no-ops against older osslili and activates once 1.7.0+ is installed. There is no release-ordering constraint.Tests
New
tests/unit/test_oslili_integration.pycovers retention and ordering of third-party licenses, the split view including the case where the same SPDX id is both own and third-party, primary-license selection, both confidence-gate directions, and pre-1.7.0 output being unaffected. ThePackageIdentifiercase drives the real coroutine viaasyncio.runrather thanpytest.mark.asyncio, since CI installs only pytest and pytest-cov.Full suite: 115 passed. Six of the new tests fail against the unfixed code, confirming they pin the regressions.
Refs SemClone/osslili#87, SemClone/osslili#78.