Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/build-cffi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -132,10 +132,6 @@ jobs:
publish:
name: Publish cffi ${{ inputs.version || '2.1.0' }} to GitLab
needs: [linux]
# Only publish when the workflow was triggered from main with a specific
# version. Manual trigger is the only entry point, so checking the ref is
# enough to gate uploads.
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/build-matplotlib.yml
Original file line number Diff line number Diff line change
Expand Up @@ -169,10 +169,6 @@ jobs:
publish:
name: Publish matplotlib ${{ inputs.version || '3.11.1' }} to GitLab
needs: build_wheels
# Only publish when the workflow was triggered from main with a specific
# version. Manual trigger is the only entry point that reaches main;
# PR runs sit on refs/pull/<n>/merge and skip this job.
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/build-numpy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -101,10 +101,6 @@ jobs:
publish:
name: Publish numpy ${{ inputs.version || '2.5.1' }} to GitLab
needs: [build_wheels]
# Only publish when the workflow was triggered from main with a specific
# version. Manual trigger is the only entry point, so checking the ref is
# enough to gate uploads.
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
4 changes: 0 additions & 4 deletions .github/workflows/build-pillow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,6 @@ jobs:
publish:
name: Publish pillow ${{ inputs.version || '12.3.0' }} to GitLab
needs: [build_wheels]
# Only publish when the workflow was triggered from main with a specific
# version. Manual trigger is the only entry point, so checking the ref is
# enough to gate uploads.
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down
71 changes: 66 additions & 5 deletions actions/publish-to-gitlab/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,9 +74,18 @@ outputs:
${{ inputs.gitlab-host }}/-/packages

uploaded-files:
description: Newline-separated list of files that twine successfully uploaded.
description: >
Newline-separated list of files that twine successfully uploaded. Empty
when running in dry-run mode (see `dry-run` output).
value: ${{ steps.upload.outputs.uploaded_files }}

dry-run:
description: >
"true" when the action ran outside of the main branch and only printed
a mock of what would have been uploaded, "false" when it actually
uploaded via twine.
value: ${{ steps.upload.outputs.dry_run }}

runs:
using: 'composite'
steps:
Expand Down Expand Up @@ -144,6 +153,15 @@ runs:
expanded=( $pattern )
if [[ ${#expanded[@]} -eq 0 ]] || [[ ! -e "${expanded[0]}" ]]; then
echo "::warning::Pattern '${pattern}' did not match any files — skipping."

dir=$(dirname -- "$pattern")
if [[ -d "$dir" ]]; then
echo " '$dir' exists but contains:"
find "$dir" -mindepth 1 -maxdepth 1 -printf ' %f\n' 2>/dev/null
[[ -z "$(ls -A "$dir" 2>/dev/null)" ]] && echo " (empty)"
else
echo " Directory '$dir' does not exist from the working directory $(pwd)."
fi
continue
fi

Expand All @@ -158,6 +176,17 @@ runs:

if [[ ${#matched_files[@]} -eq 0 ]]; then
echo "::error::No files matched the provided globs; nothing to upload."

echo "Looking for wheels/sdists elsewhere under $(pwd) (max depth 4) as a hint:"
hits="$(find . -maxdepth 4 \( -iname '*.whl' -o -iname '*.tar.gz' \) 2>/dev/null)"
if [[ -n "$hits" ]]; then
echo "$hits" | sed 's/^/ /'
echo " ^ found here instead — check the 'files' input and the artifact download path."
else
echo " None found within 4 directories of $(pwd) either."
echo " Check that the build/download-artifact step actually produced files before this action ran."
fi

echo "::endgroup::"
exit 1
fi
Expand All @@ -166,6 +195,33 @@ runs:
printf ' %s\n' "${matched_files[@]}"

echo "::endgroup::"

if [[ "${{ github.ref }}" != 'refs/heads/main' ]]; then
echo "::group::Dry run (not on main branch — no upload will happen)"

echo "dry_run=true" >> "$GITHUB_OUTPUT"

skip_flag=''
if [[ "${{ inputs.skip-existing }}" == 'true' ]]; then
skip_flag=' --skip-existing'
fi

printf 'Would run: twine upload --disable-progress-bar%s <files>\n' "$skip_flag"
printf 'Would upload %d file(s) to %s:\n' "${#matched_files[@]}" "${{ inputs.gitlab-host }}"
printf ' %s\n' "${matched_files[@]}"

echo "::endgroup::"

{
echo "uploaded_files<<EOF"
echo "EOF"
} >> "$GITHUB_OUTPUT"

exit 0
fi

echo "dry_run=false" >> "$GITHUB_OUTPUT"

echo "::group::Uploading via twine"

twine_from='twine'
Expand Down Expand Up @@ -200,7 +256,12 @@ runs:
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "**Registry endpoint:** \`${{ inputs.gitlab-host }}/api/v4/projects/${{ inputs.gitlab-project-id }}/packages/pypi\`" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "**Uploaded files:**" >> "$GITHUB_STEP_SUMMARY"
while IFS= read -r f; do
[[ -n "$f" ]] && echo "- \`$f\`" >> "$GITHUB_STEP_SUMMARY"
done <<< "${{ steps.upload.outputs.uploaded_files }}"

if [[ "${{ steps.upload.outputs.dry_run }}" == 'true' ]]; then
echo "**DRY RUN** — not on main branch, nothing was uploaded." >> "$GITHUB_STEP_SUMMARY"
else
echo "**Uploaded files:**" >> "$GITHUB_STEP_SUMMARY"
while IFS= read -r f; do
[[ -n "$f" ]] && echo "- \`$f\`" >> "$GITHUB_STEP_SUMMARY"
done <<< "${{ steps.upload.outputs.uploaded_files }}"
fi
16 changes: 15 additions & 1 deletion actions/publish-wheels/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,19 @@ runs:
fi

asset="gpl-sources/gpl-sources.tar"
url="https://github.com/${{ github.repository }}/releases/download/$tag/gpl-sources.tar"

if [[ "${{ github.ref }}" != 'refs/heads/main' ]]; then
echo "::group::Dry run (not on main branch — no release will be published)"
if gh release view "$tag" --repo "${{ github.repository }}" >/dev/null 2>&1; then
echo "Would run: gh release upload $tag $asset --clobber"
else
echo "Would run: gh release create $tag $asset --title $tag"
fi
echo "Would set GPL_SOURCES_URL=$url"
echo "::endgroup::"
exit 0
fi

if gh release view "$tag" --repo "${{ github.repository }}" >/dev/null 2>&1; then
gh release upload "$tag" "$asset" --repo "${{ github.repository }}" --clobber
Expand All @@ -144,7 +157,7 @@ runs:
--notes "Bundled GPL source packages for $tag."
fi

echo "GPL_SOURCES_URL=https://github.com/${{ github.repository }}/releases/download/$tag/gpl-sources.tar" >> "$GITHUB_ENV"
echo "GPL_SOURCES_URL=$url" >> "$GITHUB_ENV"

- uses: actions/setup-python@v5
with:
Expand All @@ -161,4 +174,5 @@ runs:
ARTIFACTS_PATH: ${{ inputs.artifact-path }}
GPL_SOURCES_URL: ${{ env.GPL_SOURCES_URL }}
GPL_SOURCES_DESCRIPTION: ${{ inputs.gpl-sources-description }}
DRY_RUN: ${{ github.ref != 'refs/heads/main' && 'true' || 'false' }}
run: python3 ci_scripts/update_doc.py
39 changes: 28 additions & 11 deletions ci_scripts/update_doc.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
truth; it never touches docs/packages/*.md or index.md directly.
"""

import difflib
import os
import re
import string
Expand All @@ -30,6 +31,7 @@
ARTIFACTS_PATH = os.environ.get("ARTIFACTS_PATH", "dist")
GPL_SOURCES_URL = os.environ.get("GPL_SOURCES_URL")
GPL_SOURCES_DESCRIPTION = os.environ.get("GPL_SOURCES_DESCRIPTION", "").strip()
DRY_RUN = os.environ.get("DRY_RUN", "false").strip().lower() == "true"


def find_wheel_file(path):
Expand Down Expand Up @@ -221,25 +223,40 @@ def main():
comment = render_gpl_sources_comment()
yaml_path = DOCS_DIR / f"{slug}.yaml"
is_new = not yaml_path.exists()
old_content = None if is_new else yaml_path.read_text()

if is_new:
yaml_path.write_text(
render_new_yaml(slug, source_code, license, version, patch_dir, comment)
)
new_content = render_new_yaml(slug, source_code, license, version, patch_dir, comment)
else:
content = yaml_path.read_text()
package_data = yaml.safe_load(content) or {}
updated = append_version(
content, package_data, version, license, patch_dir, comment
package_data = yaml.safe_load(old_content) or {}
new_content = append_version(
old_content, package_data, version, license, patch_dir, comment
)
if updated is None:
if new_content is None:
print(f"{slug} {version} is already documented; nothing to do")
return
yaml_path.write_text(updated)

branch = f"github-actions/{'add' if is_new else 'update'}-doc-for-{slug}"
pr_title = f"docs: {'add' if is_new else 'update'} {slug}"

if DRY_RUN:
print("[dry-run] Not on main branch — no branch, commit, or PR will be created.")
print(f"[dry-run] Would write {yaml_path}:")
diff = difflib.unified_diff(
(old_content or "").splitlines(keepends=True),
new_content.splitlines(keepends=True),
fromfile=str(yaml_path) if old_content is not None else "/dev/null",
tofile=str(yaml_path),
)
sys.stdout.writelines(diff)
if is_new:
print(f"[dry-run] Would add '{slug}' to {PACKAGES_FILE}")
print(f"[dry-run] Would open PR '{pr_title}' from branch '{branch}' against main")
return

yaml_path.write_text(new_content)
configure_git_identity()

branch = f"github-actions/{'add' if is_new else 'update'}-doc-for-{slug}"
git_run("switch", "-c", branch)
git_run("add", str(yaml_path))

Expand All @@ -259,7 +276,7 @@ def main():
"--base", "main",
"--head", branch,
"--reviewer", "threexc,justeph",
"--title", f"docs: {'add' if is_new else 'update'} {slug}",
"--title", pr_title,
"--body",
"Automatically generated PR to document a newly published wheel. "
"Please review it carefully before merging.\n\n"
Expand Down
17 changes: 9 additions & 8 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,10 +144,6 @@ workflow needs is `publish-wheels`, which performs the following steps:
publish:
name: Publish numpy ${{ inputs.version || '2.5.0' }} to GitLab
needs: build_wheels
# Only publish when the workflow was triggered from main with a specific
# version. Manual trigger is the only entry point, so checking the ref is
# enough to gate uploads.
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
permissions:
contents: write
Expand Down Expand Up @@ -231,11 +227,16 @@ user experience when consuming wheels from RISE's package registry.**

## Releasing a Wheel

The `publish-to-gitlab` action does not run unless the workflow is triggered
from main. This is intentional, and is meant to ensure that only those workflows
The `publish-to-gitlab` and `publish-wheels` actions only perform their real
side effects (twine upload, GPL sources release, docs PR) when the workflow is
triggered from `main`. On any other ref they print a dry-run instead — the
resolved file globs, the twine command that would have run, and the branch/PR
title `update_doc.py` would have used — without uploading anything or opening
a PR. This is intentional, and is meant to ensure that only those workflows
which have been fully tested, reviewed, and merged are used to build and push
packages. Following the merge of a PR, the workflow(s) must be re-triggered from
the `main` branch in order to release the wheels to the package registry.
packages, while still letting a PR run demonstrate what publishing would do.
Following the merge of a PR, the workflow(s) must be re-triggered from the
`main` branch in order to actually release the wheels to the package registry.

## Other Workflow Tips and Tricks

Expand Down
Loading