feat(deploy): T399695 - Initial Deploy config/setup wizard (Web and CLI) #879
Conversation
lorenjohnson
force-pushed
the
deploy-setup-script
branch
from
32b7d79 to
8a5ef0a
Compare
lorenjohnson
changed the title
lorenjohnson
force-pushed
the
deploy-setup-script
branch
2 times, most recently
from
0681785 to
fee35da
Compare
lorenjohnson
force-pushed
the
deploy-setup-script
branch
from
8c3e257 to
abc0dfd
Compare
lorenjohnson
changed the title
lorenjohnson
force-pushed
the
deploy-setup-script
branch
3 times, most recently
from
e8b9376 to
c2fb04f
Compare
lorenjohnson
force-pushed
the
deploy-setup-script
branch
from
a8cee84 to
430017d
Compare
… structure shell script code for extension/maintenance.
Unfortunately, MediaWiki still stores the DB password in plain text in the |
Is not considered scope of this PR, correct? |
| echo "⚠️ Unsupported package manager. Please install Git manually." | ||
| exit 1 | ||
| fi | ||
| } |
There was a problem hiding this comment.
There is no error handling and all outputs in install_git deleted. This makes it hard to debug if something goes wrong.
There was a problem hiding this comment.
I can defend this more somewhere, but the long and short of it is I am actually ok with that for now.
There was a problem hiding this comment.
I am not :)
Not handling all the errors is fine with me. But I would really prefer having them show up somewhere. Otherwise debugging becomes hell.
There was a problem hiding this comment.
Did you have a recommendation here in terms of code? I'm assuming you have looked at /tmp/wbs-deploy-setup.log, the results of any commands above will be in there. Otherwise I am actually not sure what specifically you are pointing to.
There was a problem hiding this comment.
My bad. I think I misread the piping going on there. If the stderr output is in the logfile, I am happy. Thanks for clarifying.
lorenjohnson
force-pushed
the
deploy-setup-script
branch
from
df41a6a to
ba65f6e
Compare
deploy/setup/scripts/launch.sh
Outdated
| # Internal re-entry flag used when ran detached | ||
| LAUNCH_ONLY=$([ "${1-}" = "--launch-only" ] && echo true || echo false) |
There was a problem hiding this comment.
Is the comment still up to date? This does not look like a re-entry flag, does it?
There was a problem hiding this comment.
It is a re-entry flag, but it could be renamed if something is more clear.
There was a problem hiding this comment.
I see now. Interesting flow. Not obvious on the first sight, but makes sense. Did you consider splitting this into two script files?
There was a problem hiding this comment.
I will consider it further.
| run "sudo apt-get update -y" | ||
| run "sudo apt-get install -y --no-install-recommends ca-certificates curl gnupg" | ||
| run "sudo install -m 0755 -d /etc/apt/keyrings" | ||
| run "curl -fsSL https://download.docker.com/linux/$OS_ID/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg" |
There was a problem hiding this comment.
I'd prefer to use docker.io from the ubuntu / debian repos. Is there a reason to not?
There was a problem hiding this comment.
I preferred this as I was sure to get the more recent versions and the same code runs and works regardless of dist/version.
There was a problem hiding this comment.
I have the strong opinion that it is not good practice to get packages from external sources if distro repos provide what is needed. Is docker.io too old in any of the distro repos?
There was a problem hiding this comment.
Yes, my understanding from you and from testing was that at least one of the Debian repos for one of the supported Debian versions was out of date (only providing docker 1.x)
There was a problem hiding this comment.
Yes, docker-compose was the problem on debian. Now with trixie being stable, we also have compose v2 natively on debian https://packages.debian.org/search?suite=trixie&arch=any&searchon=names&keywords=docker-compose
deploy/setup/scripts/web-config.sh
Outdated
| if $LOCALHOST; then | ||
| echo It is now safe to close this terminal session. | ||
| echo | ||
| fi |
There was a problem hiding this comment.
Why is this only safe in the localhost case?
There was a problem hiding this comment.
I think this condition is actually wrong, it should be the opposite of what it says.
The setup tool was to this point scoped to only two scenarios: 1) running via curl on a newly minted VPS, or, 2) running locally on your own computer for testing or dev.
In the VPS scenario this script is ran in the background via nohup, vs in the local scenario it is ran interactively and everything goes to the terminal directly. It is in the case of it being ran directly and not sent out via nohup that this message should appear.
There was a problem hiding this comment.
Sorry, I had spoken too quickly in my previous response. Current ^^ response is accurate.
…story, also replaces cd with pushd/popd in launch_wikibase, moves safe to exit terminal message
…sting setup container before trying to start new one
…uery.wikibase.test defaults for localhost configuration
…r passing and _logging usage
… validation issues, adds more host name option notes
…h_wikibase to launch_deploy
To test feature see README and follow the instructions there:
https://github.com/wmde/wikibase-release-pipeline/tree/deploy-setup-script/deploy/setup/README.md
Related Phabricator ticket: T399695
Remaining TODOs
index.etaLAUNCH_ONLY/ re-run handling in launch.shconsider incorporating the common words dictionary lookup here. Have a "Generate" link for passwords or keep
auto generated if ignored?
deploy/README.mdthrough todeploy/setup/README.mdTesting notes
./setup.shdirectly from the repo’sdeploy/setupdirectory skips Git install and repo clone.To test those steps locally you have a few options:
NOTE: The full debug log is available for each run at
/tmp/wbs-deploy-setup.logTroubleshooting section notes
"Invalid Certifcate" warnings:
Let's Encrypt rate limiting for *.nip.ip domains used for the setup installer means we may need to an "invalid cert" warning arising from self-signed SSL/TLS certificates when trying to access the page. This warning will also always appear when using --local option as all certificates in that scenario are self-signed, including as well for the WBS services.
If for any reason you repeat the setup numerous times using the same values for Wikibase Public Host and/or Query Service Public Host, you may also see this invalid certificate warning for your site. If you want to continue to use those host names you will need to wait (how long?) before trying again so the certficate provider (Let's Encrypt) will unblock certificate provision for your designated hosts.
In the setup case it is safe and fully functional to bypass the warning and continue. See https://docs.vultr.com/how-to-bypass-the-https-warning-for-self-signed-ssl-tls-certificates for some guidance on how to do so.
High level narrative introduction of the options available for setting up Wikibase Suite Deploy (DRAFT)
Possibly a script for a short Wikibase Suite Deploy intro video?
Wikibase Suite Deploy is the easiest way to get a full Wikibase environment up and running — all the moving parts integrated, configured, and ready to use.
There are two main ways to install it: the manual method, and the setup helper. Both get you to the same place: a fully working Wikibase Suite instance. The manual method is documented in the
deploydirectory’s README and walks you through each step yourself. The setup helper, found in thedeploy/setupdirectory, does all of those steps for you — and more.With the setup helper, you don’t even need to install Git or Docker yourself, or clone the repository. The script will handle that automatically if they aren’t already installed. It also provides:
Once you’ve entered your configuration, the setup helper will launch your Wikibase Suite for the first time, let you know when it’s available, and tell you if there are any issues to fix.
The manual method works equally well — it’s just more hands-on and easier to make mistakes. It can also be useful for debugging if the setup helper isn’t working for some reason. But for most users, the setup helper is the recommended path.
At this stage, if something fails during installation — whether you’ve used the manual method or the setup helper — the safest way to recover is to delete your VPS instance, create a fresh one, and start again. That avoids tricky issues with partially completed installs or SSL certificate conflicts.
You can also run Wikibase Suite Deploy locally. This is only partially supported because it requires HTTPS for all services, and generating a valid SSL certificate for
localhostisn’t straightforward. Locally, you’ll see a browser warning that the certificate is invalid — you can override this, and in most cases your browser won’t ask again. But QuickStatements won’t work locally at this time because it can’t authenticate without a valid certificate.So:
These instructions are all about getting to your first running instance. For day-to-day operations — things like backing up or resetting your data, starting and stopping services, or debugging in production — you’ll find more details in the main
deployREADME.One last note: the configuration you enter during setup is for initial deployment only. It’s written to the
.envfile in the deploy directory. Changing these values after the fact won’t reconfigure your running instance — in some cases it will do nothing, in others it may cause unpredictable results.If you get stuck, the Wikibase Suite team is here to help — reach out, and we’ll work with you to get your instance up and running.
Related Feature Stories
Implemented
Won't be implemented