Changes to support splitting CI build into multiple containers. - #1889
Draft
ShixinWu16 wants to merge 14 commits into
Draft
Changes to support splitting CI build into multiple containers.#1889ShixinWu16 wants to merge 14 commits into
ShixinWu16 wants to merge 14 commits into
Conversation
ryanrath
added
Category:Infrastructure
ShixinWu16
force-pushed
the
file_updates_for_refactoring_docker_files
branch
from
July 29, 2024 15:03
6ee958a to
16e723d
Compare
jpwhite4
changed the title
jpwhite4
marked this pull request as draft
October 28, 2025 14:42
ryanrath
force-pushed
the
file_updates_for_refactoring_docker_files
branch
2 times, most recently
from
October 30, 2025 18:57
5b5487a to
cda8844
Compare
-
tests/artifacts/xdmod/etlv2/configuration/output/normalized_table_definition.json:
- Updated to use the new docker network name.
- tests/ci/bootstrap.sh
- removed the database related setup.
- tests/ci/scripts/xdmod-setup-start.tcl:
- Updated to use the new docker network naeme.
- tests/playwright/Docker/docker-compose.yml
- Added the `mariadb` container
- Added a new `05-skipcache.cnf` file that disables the feature that
makes mariadb use ip addresses vs. dns names as our code assumes
that our db users are `username@hostname`.
- Added a modified version of `/root/bin/services` that does not have
the db related commands in it. Of note, the order of the services
stopping / starting is intentional. If httpd is the first thing
stopped / started an error occurred on start about port 80 still
being in use thus causing restarts to fail.
- Added a `my.cnf` file that is mapped into the xdmod container as
`/root/.my.cnf` since we run a few raw `mysql` commands assuming we
are connecting to localhost.
ryanrath
force-pushed
the
file_updates_for_refactoring_docker_files
branch
from
July 14, 2026 15:11
f4a3a50 to
fe91b31
Compare
This commit adds the ability to run / test the upgrade path using the newly seperated docker compose setup ( having a seperate db container as opposed to having it installed in the xdmod container ). Since the upgrade path assumes that we have a populated db I've added an export of the db from the built 11.0.3 docker image that is imported on startup of the mariadb container. I'm not completely wedded to doing things this way, but it was easy to implement and fulfills the need. We could also do some pre-processing steps of: - export the dbs from the xdmod container - import these dbs into the mariadb container - ... - profit? This would obviously save us the space, but could add more time to the build. I'll do some testing and update the eventual PR with the results.
- Got rid of having a docker-compose file for both fresh_install && upgrade and consolidated it to one docker-compose file. - Some minor changes to the xdmod Dockerfile to clean up and remove unneeded files / steps - Some minor changes to `.circleci/config.yml` to ensure that we're not using the on image DB anymore. - Updates to `tests/ci/bootstrap.sh` that ensure we start with an empty db when we're doing a fresh_install.
Member
|
As per our conversation: Since the mysql gzip files are output artifacts from the build we should not add them to the source repository. The plan wil be to modify the docker image creation script that is in ccr-private so that it generates two images (1) the image that contains the current release of XDMoD with webserver, php, but not the database and (2) the image that contains the database. This pull request will then be modified to use these two images in the CI build. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Changes to support XDMoD database on an external host
Added a check for if the user was created
Added a question to get XDMoD host
Changed from using bootstrap to entrypoint (in create docker images repo)
#### Updated collation from utf8_general_ci to utf8_unicode_ciconfiguration/etl/etl_tables.d/*tests/artifacts/xdmod/etlv2/configuration/input/*tests/artifacts/xdmod/etlv2/configuration/output/*tests/unit/lib/ETL/DbModel/*Update ryanrath: The above changes were merged in a different PR and as such have been removed from this PR
Motivation and Context
We wanted to split up the docker containers so that each module has their own container.
This would allow us to share our docker images without having to share everything.
Tests performed
Ran all automated tests, changed tests to match new collation sets
Checklist: