If you don't have any sites configured on Wagtail (very possible if you're running headless) and don't have localhost in your Django ALLOWED_HOSTS, you'll get an error. But you can avoid adding localhost to your ALLOWED_HOSTS by setting up a site.
I figured this out by raising an issue against wagtail, maybe this should go somewhere in the readme?