Hugo on the go

Welcome! Here I’m attempting to chronicle my attempts (and many errors) at being able to update my Hugo site from my tablet. It took a while to figure it out, but I think I’ve sorted out a proper workflow to get it to work without a hitch. I’m doing this on my Samsung Galaxy Tab S6 Lite Wifi (SM-P610), but you can certainly get it to work on any other device, including a phone, if you so choose; you can do this in a number of ways.

Update 03 Jun 2022: I added a “Caveats” section re: syncing large files, for example, below

The Basic Idea

Here is the basic idea:

• I want to be able to work on my website on the go, e.g., on my tablet.
• I want to be able to regenerate the site whenever I want, working from a current snapshot of the site
• I want to be able then to push those updates then to the site.

This is a very different paradigm than I’m used to; I am used to working with WordPress, where it does all the “heavy lifting,” so to speak, and I just get my writing done. Generating a website in Hugo involves a little more setup on the front end, but once that setup is done (and it really should only need to be done once), everything is real straightforward. Even though it’s not “instantaneous” like WordPress, I’m really enjoying it.

Prerequisites

Prerequisites include:

• A VPS (Virtual Private Server) or other webspace to which you can upload the site
  • Ideally, you’ll have an existing Hugo site that you’re going to update and rebuild
  • If you don’t know what Hugo is, please see their excellent quickstart guide to get up and running.
  • A working Hugo installation on your VPS; see here for help with that.
    • You can also install Hugo on the tablet in Termux using pkg install hugo but I wouldn’t recommend that, as the permissions ended up getting all corrupted when I generated my site.
• A device besides your laptop/desktop upon which to set up this workflow
• A way to mirror your whole site on that device
  • This can include syncing solutions like NextCloud, SyncThing, vel sim., or if you don’t have access to any of those, I’m sure Google Drive will work.
  • I’m using NextCloud myself to keep it synced, but whatever works easiest for you.

Let’s do it

I played around with a lot of options before arriving on this, but hey, as I said, make the workflow your own if you like. This is just what works best for me.

First, make sure your site is how you want it. I’m assuming you’ve generated a Hugo site and have the general directories set up (content/, content/posts, &c.). So we’re going to go with that. If you haven’t, check out the link for the Hugo quickstart guide above, and pop back in once the site skeleton is ready.

More prerequisites: homedir and other permissions for ssh

So one thing that really tripped me up was, of all things, permissions on the home directory of my hugo user and even the .ssh directory that holds the public key.

• Groups shouldn’t have write access to the user’s homedir: chmod go-w /home/user (honestly you can just do it as chmod 700 if you like)
• The hugo user’s (or your admin’s, or any user, really) .ssh directory needs to be set to 700 (rwx------ if you ls -l the directory): chmod 700 ~/.ssh
• The user’s .ssh/authorized_keys needs to be chmod 600: chmod 600 ~/.ssh/authorized_keys
• Importantly, the hugo user (or your admin) needs to be set as the user:group for the homedir: chown -R [user]:[user] [homedir]
  • This really tripped me up. It’s a big deal!

Again, for whatever reason, ssh is really picky about permissions, and this is a new thing for me. So I had to learn the hard way and look at /var/log/auth.log and such to figure out what was going on. Figured I’d save y’all the trouble.

Rsync time: uploading your site to your VPS

Once the permissions are set up correctly, you’ll want to actually upload the site.

So as I mentioned, I am using NextCloud to sync the site, so it’s available on my tablet as well due to the NextCloud connection. I set it up so my tablet is syncing my hugo skeleton directory. I run LineageOS (a variant of Android), too, so I’m using Termux for part of this. Termux is a package available on F-Droid which lets you run a terminal emulator and install packages (.deb packages, using similar syntax to apt, but pkg instead).

For this to work properly you’ll need the Termux API package as well. You’ll also need to have set up accessing “external storage” (i.e., your tablet/phone’s SD card and internal memory). Here’s a guide to setting that up.

• Make sure your NextCloud or sync solution has a copy of the site available on your tablet/phone. E.g., run sync in your solution of choice and make sure it’s accessible.
• Fire up Termux and cd to the storage/ directory off of your homedir.
• If you have NextCloud installed, its local copy of your synced NextCloud files should be available off of shared/Android/media/com.nextcloud.client/files or a similar path.
• cd over to that path.

Now we’re going to rsync the contents of that directory to our server. If you don’t have rsync, install it using your package manager.

• Since I’m using Termux, I’d install it using pkg install rsync but whatever works best for you.

In my case, I’m using YunoHost and the my_webapp app to run Hugo. The app is just a plain directory viewable to the web, so you can put whatever you want in there. I linked this elsewhere, but this guide was really useful in setting everything up for Hugo on YunoHost. Otherwise, just rsync to the directory where your webserver files live.

The command I use is: rsync -avz -e "ssh -i [keyfile]" --progress . [username]@[host]:[webserver_directory]

• It’s really best to set up SSH public key authentication if you can. YunoHost has a nice tutorial on how to set this up. And I found others such as this one useful as well. You can use password authentication if you like, but it’s better and more secure not to. The -i [keyfile] part is used for that.

Personally I rsync to a temporary hugo/ directory just shy of the webserver directory, because honestly you’re gonna want to keep it somewhere temporary until you can build the site.

You can use sftp as well for this, but rsync does a way better job and is way faster, IMHO.

Generating the site

After that, once it’s rsync’d, ssh into the site: ssh [-i keyfile] user@host

I’m assuming you’ve generated your hugo site before. If you have, cd into the public/ directory and rm -rf everything there, since we want a clean directory to build into.

• If you haven’t, per the Hugo quickstart guide, find a suitable directory on your laptop or wherever the Hugo site normally lives, and type hugo new site [sitename] and it will create a new site skeleton in the [sitename] directory off of your current directory. Seriously, that quickstart guide will get you up and running.

Go into your hugo site’s directory on the server, and run hugo

It will generate the site for you, give you some stats, &c., and then your site will be generated in public/ off of that directory.

Copying the site to make it live

Once public/ is filled with files, your site is generated.

N.B.: You may, just to be safe, want to chmod 755 the public/ directory, just in case there are permissions issues; on my server, the www/ directory and its files have to be 755 to be viewable (I think they need the -x bit for whatever reason).

All that remains is to copy the contents of public to your webserver directory. For me, the command resembles this: cp -R public/* www/

After that, I cd out of the hugo/ temp directory and rm -rf it.

And that’s it! Go to your site and it’ll be live and updated!

Scripting/automation

So I think at least some of this can be scripted and made non-interactive. Depends on whether you have a passphrase for your SSH public key. I didn’t put a passphrase on the SSH public key for my hugo user, since 1) password authentication is disabled and therefore 2) you can’t log in without a key anyway. so it makes it easier to script.

I’m working on and thinking about how to script this. I can write the script as a bash or sh script and run it, but some commands need to be passed along to the remote server, and I’m not 100% sure how to do that. So that is a work in progress. But the manual workflow will work quite nicely (and has).

Stray observations

• Again, SSH is picky about permissions. Make sure yours are correct so you won’t have login issues.
• rsync is your friend for uploading the site!
• Make sure that when you modify a file for your site, it’s synced and you’re then grabbing the newest, most current version of the site.

Other than that, just have fun with this, and congratulate yourself for being able to work on your Hugo site on whatever platform you have, mobile or laptop!

Caveats

One thing to keep in mind is that if you have any large files (mp3s, zips, &c.), they’ll get synced as well. I found that out the hard way with my “Erstwhile DJ Days” post, which has about 280Mb of mp3s. NextCloud was syncing them not only from the content/ directory where the post lived, but also the public/ directory when the site is generated.

To fix this, I had to go into the NextCloud desktop sync app’s settings and tell it not to sync *.mp3 and *.zip files. That means I’m going to keep the public/ directory intact in the www/ directory, and just rsync it whenever I update the site. That way, I don’t have to re-upload the mp3s every time with rsync either.

We’ll see how this goes in practice. I still need to write a script for the tablet to do all of this; I have a rudimentary script on the laptop to take care of it, but half the time I end up doing it manually anyway on the laptop.

It was a little goofy to have to do a workaround like this for one post, but hey, I want to keep my site “lean and mean” and not have to unnecessarily sync gigs of data (hey, 200Mb adds up; 5x is 1Gb roughly, so that’s no small amount, and it takes time!). I’d rather take the time to keep it from happening than take the time to keep reuploading the files.

Addendum re: large files

I found a good way to deal with the mp3s/media/whatever issue. Now if your site is all text and/or just a few images, and fairly lightweight, don’t worry about doing all this. I’m just doing this so I don’t have to keep resyncing all this data every time. I just want to have to deal with the main files of the site and leave the mp3s &c. intact.

Prerequisites: upload a copy of your entire hugo directory to the hugo user’s home directory. This means you’ll essentially have two copies of your site, which, if it’s lightweight, shouldn’t be an issue. Use rsync or whatever you prefer, just have the directory up there somewhere.

New workflow:

• ssh into the VPS with the hugo user’s account, howsoever you do this
• cd to the “copy” of the hugo directory
• Delete all files in the directory besides the media files you want to keep:
  • I used the following command: find . ! -name \*.mp3 ! -name \*.zip -delete
  • ! naturally means “not”; this command means “find all files and delete only the ones not matching the -name expressions.
  • Note the \ characters to escape the asterisks; this is necessary on most shells.
• Exit out of the ssh session
• Rsync the hugo directory from your local NextCloud synced copy to the temp directory: rsync -avz -e "ssh -i [keyfile]" --progress . [user]@[host]:/hugo/temp/dir
• ssh back into the hugo user’s account
• Go into the “copy” of the hugo directory and make sure public/ is empty rm -rf public/
• run hugo to generate the site
  • important: then chmod 755 public as we’re going to rsync public/ to the www/ directory and for whatever reason it needs to be executable (744 won’t work, at least for my server setup).
• Delete your whole www/ directory where the hugo blog files reside
• From the hugo copy’s public/ directory, rsync to the www/ directory: rsync -avz . ../../www
  • Obviously your paths may vary, so adjust accordingly.

I need to write a script for this on the server, somewhere in the hugo user’s homedir, to automate this whole thing. But it may involve scripting locally and remotely.

When syncing from the laptop, I shouldn’t need to do any of this… but I suppose I could do the same find command to the www/ directory before syncing.

Conclusion and Acknowledgements

Again, thanks so much to my friends on Mastodon who helped me think this through: LJ(writes), cadadr, emacsomancer, FiXato, & probably others I’m forgetting (ignoscite mihi!).

As with other blog posts, this is a work-in-progress and may have further edits, but I hope the information is useful to you if you’re thinking of doing this yourself!