Skip to content

fix(pipelines/tiflash): release-6.5 IT runtime needs mysql client - #4850

Merged
ti-chi-bot[bot] merged 1 commit into
mainfrom
fix/tiflash-r65-it-runtime-with-mysql
Jul 29, 2026
Merged

fix(pipelines/tiflash): release-6.5 IT runtime needs mysql client#4850
ti-chi-bot[bot] merged 1 commit into
mainfrom
fix/tiflash-r65-it-runtime-with-mysql

Conversation

@kumailf

@kumailf kumailf commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

Follow-up to #4849. Rewriting untagged tiflash-ci-base to cd/builders/tiflash:v20231106 unblocked registry pulls, but fullstack tests exec into that container and need mysql client:

/bin/sh: mysql: command not found

Old hub.pingcap.net/tiflash/tiflash-ci-base included mysql (see jenkins/Dockerfile/release/linux-amd64/tiflash_ci_base). Builder images do not.

Change

  • Pull ghcr.io/pingcap-qe/bases/tiflash-base:v1.9.1
  • Locally build tiflash-ci-base-local:with-mysql with yum install -y mysql (same recipe as historical Dockerfile)
  • Use that image for compose rewrites of tiflash-ci-base / tics

Keeps failpoint fallback from #4849.

Test plan

…time

Builder image v20231106 has no mysql client, so fullstack run-test.sh fails
with "mysql: command not found" after rewriting tiflash-ci-base. Rebuild the
historical runtime from ghcr.io/pingcap-qe/bases/tiflash-base:v1.9.1 with yum
install mysql (same as jenkins tiflash_ci_base Dockerfile).

@ti-chi-bot ti-chi-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have already done a preliminary review for you, and I hope to help you do a better job.

Summary:
This PR addresses the missing mysql client in the tiflash-ci-base image used during integration testing for TiFlash release-6.5. Instead of using the previous untagged image, it switches to a specific base image ghcr.io/pingcap-qe/bases/tiflash-base:v1.9.1 and locally builds a new image with mysql installed via yum. The approach is straightforward and clearly solves the missing client issue. The patch is concise and focuses on the relevant pipeline Groovy file with reasonable inline Dockerfile usage. Overall, the change is well scoped but can be improved for robustness and maintainability.


Critical Issues

  • Potential docker build caching issues (pipelines/pingcap/tiflash/release-6.5/pull_integration_test.groovy, line ~280)
    • The Docker image tiflash-ci-base-local:with-mysql is built every run without cache invalidation or tagging by commit/version.
    • This might cause stale builds or unnecessary rebuilds, which is inefficient and could lead to confusion if the base image updates but the local image does not.
    • Suggestion: Add a unique tag incorporating the base image digest or a build timestamp, or use a build cache strategy to ensure builds are only triggered when necessary.

Code Improvements

  • Hardcoded image tag and inline Dockerfile (pull_integration_test.groovy, lines ~278-288)

    • Embedding the Dockerfile inline in the Groovy script makes the pipeline harder to maintain and review.
    • This approach also mixes concerns: pipeline logic and image definition.
    • Suggestion: Extract this Dockerfile into a dedicated Dockerfile.mysql or similar in the repo, and build from that file. This improves clarity and allows independent testing of the image build.
  • No error handling for docker build failures

    • The docker build command is run inline but there is no explicit error handling or retry logic.
    • If the build fails (network, yum repo issues), the pipeline will fail but may not produce clear logs.
    • Suggestion: Add error checking or capture logs clearly. Optionally, retry the build or fail fast with an informative message.
  • Inconsistent environment variable naming

    • The original variable TIFLASH_IMAGE is replaced by TIFLASH_IT_BASE_IMAGE in the environment block, but later TIFLASH_IMAGE is reassigned locally.
    • This can be confusing for readers and maintainers.
    • Suggestion: Keep consistent variable naming or clearly comment the difference between base image and final test image.

Best Practices

  • Missing comments on the Docker build snippet

    • Although there is a short comment explaining the reason for the build, the Dockerfile snippet itself lacks explanation.
    • Adding a comment within the Dockerfile snippet would clarify the purpose of the yum install -y mysql step.
  • Testing coverage not confirmed

    • The PR description mentions manual test plan steps but no automated tests or pipeline checks to verify the image contains mysql.
    • Suggestion: Add a quick smoke test after build (e.g., docker run --rm tiflash-ci-base-local:with-mysql mysql --version) to verify installation succeeded.
  • Style: shell script quoting and heredoc usage

    • Using a heredoc with <<EOF is fine, but consider using <<-'EOF' or quoting the EOF delimiter to prevent unwanted variable expansion if any variables are introduced later.
    • This is minor but can prevent subtle bugs.

Summary of Suggested Fixes

// Extract dockerfile to pipelines/pingcap/tiflash/release-6.5/Dockerfile.mysql
// with content:
// FROM ghcr.io/pingcap-qe/bases/tiflash-base:v1.9.1
// RUN yum install -y mysql

timeout 600 docker pull "${TIFLASH_IT_BASE_IMAGE}"

// Use a unique tag with timestamp or digest
def mysqlImageTag = "tiflash-ci-base-local:with-mysql-${env.BUILD_ID}"
sh "docker build -t ${mysqlImageTag} -f pipelines/pingcap/tiflash/release-6.5/Dockerfile.mysql ."

// Optional test to verify mysql client
sh "docker run --rm ${mysqlImageTag} mysql --version"

// Use consistent env var for downstream usage
TIFLASH_IMAGE = mysqlImageTag

This will improve maintainability, caching, and robustness of the fix.

@dillon-zheng dillon-zheng left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot Bot added the lgtm label Jul 29, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dillon-zheng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot

ti-chi-bot Bot commented Jul 29, 2026

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2026-07-29 06:43:30.241079861 +0000 UTC m=+1991996.277174917: ☑️ agreed by dillon-zheng.

@ti-chi-bot ti-chi-bot Bot added the approved label Jul 29, 2026
@ti-chi-bot
ti-chi-bot Bot merged commit 8665142 into main Jul 29, 2026
7 checks passed
@ti-chi-bot
ti-chi-bot Bot deleted the fix/tiflash-r65-it-runtime-with-mysql branch July 29, 2026 06:46
ti-chi-bot Bot pushed a commit that referenced this pull request Jul 29, 2026
…4852)

## Summary
Follow-up after #4849 / #4850.

`tidb-ci/cluster.yaml` on release-6.5 uses `qa/tidb:...-failpoint`. The
previous logic only skipped `fail-point-tests` when the failpoint image
**pull failed**. OCI still has `tidb-server:release-6.5-failpoint`, so
pull succeeds, but the image does not expose `enableTestAPI`:

```text
Error: get fail/github.com/pingcap/tidb/server/enableTestAPI
```

## Change
- Always fall back `TIDB_FAILPOINT_IMAGE` → regular `release-6.5` TiDB
image
- Always strip `./run-test.sh tidb-ci/fail-point-tests &&` from `run.sh`
- Other suites (`fullstack-test`, async_grpc, collation, …) still run

## Test plan
- [ ] Merge and `/test pull-integration-test` on pingcap/tiflash#10993
- [ ] Logs show WARN skip fail-point-tests
- [ ] `tidb-ci` proceeds to fullstack-test without enableTestAPI error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

2 participants