Robot Unsalted's picture

Hi All,


This is my follow up from this thread: https://www.turnkeylinux.org/comment/55352#comment-55352

 

I  wanted to upgrade my nextcloud turnkey linux container on proxmox to the latest version 17.2. I am a new to all this, so apologies in advance.

I did run the command previously suggested and found that I am running version 17.1.

root@nextcloud /# turnkey-version
turnkey-nextcloud-17.1-bullseye-amd64

For some further context, I am running Nextcloud container on a Proxmox homelab server locally and the server isn't internet accessible. I want to stick to the "official" way of upgrading to the latest version of Nextcloud and don't want to attempt a in-place upgrade (and the maintainence that might come with that), especially now that the container has data that I would consider "critical."

Now, I don't completely understand how I can upgrade without losing the customization (HTTPS access, etc.), apps, commands, data (photos, videos, and phone backups), and IP address (which currently is 192.168.1.101). Hoping that the IP address is the same so I don't need to change it for all the clients and will work on just having them point to an external domain/internal IP address (that is another project for the future).

Forum: 
Timmy's picture

But one question I have right off the bat - can you back up the nextcloud VM locally on your proxmox server?

Barring that, can you copy the data to some other device?

Any method of cloning the data to somewhere else will make the upgrade a lot less nerve-wracking.

 

I'm currently stuck on 23 myself- been planning do a move as well once TKL had a PHP 8 offering.

Jeremy Davis's picture

Hopefully, I've covered your general questions in my reply below, but to give you an explicit response:

Yes, please do do a Proxmox backup! As noted, I often use TKLBAM too (for remote/off-site backups) but local snapshots (i.e. the whole OS) are probably best for this sort of thing.

Robot Unsalted's picture

Hi Timmy,

Yes, you can backup the nextcloud VM/container locally on your proxmox server. On your proxmox console, you can click on the container > backup > Backup Now > then back up it to your local machine by clicking "local."

Yes, you can copy the data elsewhere. On your proxmox console, you can click on Datacenter > Storage > Add > The type of storage you want to add. I personally am using two raspberrypis with external hard drives attached running OpenMediaVault (OMV) with NFS shares (which likely isn't the best way to do it). But after adding those to the console, you can click on the container/VM you want to backup > Backup > Backup now > choose the storage you can to backup in as well as other settings.

In terms of making changes to the VMs/containers less nerve wrecking:

you can also schedule these backups. On your proxmox console, you can click on Datacenter > Backup > Add > Setup up backups based on the time schedule and containers/VMs you want to backup.

You can also take snapshots in proxmox. On you proxmox console, click on the container/VM you want to snapshot > Snapshots > Take Snapshot

Hope this helps

Timmy's picture

Robot- I believe you misunderstood my question. I was asking you if you were capable of backing up your instance prior to attempting any upgrades. It looks like you have updated your original post but as I recall, there was no mention of backups or capacity to backup data. I know the pain of losing important data so I always ask what the backup solution is before risky operations.

As for myself, I operate a full Proxmox Backup Server instance instead of performing local storage backups or any sort of network-attached storage.

Jeremy Davis's picture

Apologies that I didn't get a chance to post back yesterday. I'm so close to getting v18.0 stable release of Core and TKLDev (and hopefully at least LAMP as well). Currently I'm suffering death by a thousand paper cuts, but fingers crossed, I should have something published within the week (although I'll temper than with the observation that I've been saying "within the new few weeks" for well over a month now...). Anyway...

As one last aside before I get to the point, this particular issue (ease of updating/migrating) is something that I hope to improve during the life of v18.x.

Ok, so as I mentioned, there are 2 paths to updating Nextcloud (in no particular order - and generally applies to all of our appliances):

  1. One is to manually update things on your current server (i.e. Nextcloud and dependencies; perhaps even upgrade the OS itself). That will have the advantage of meaning that your other existing config won't change (or not too much if you upgrade OS). The downside is that it may involve a bit of work (inc some trial and error) and if you update OS, will result in something of a hybrid - not quite completely TurnKey anymore but not pure Debian either.
  2. And the other, is to migrate your data to a newer server. This has the advantage that you generally have less work to do and only migrate what you want/need.

Manually update Nextcloud and deps

This path is in theory relatively straight forward, or at least for Debian sysadmin types... If you were to put the OS upgrade aside, then generally just upgrading Nextcloud should be relatively straight forward. But the requirement of newer PHP makes it trickier than it would be otherwise.

In v17.2, we install PHP 8.1 (from third party: deb.sury.org). Whilst it's not that hard to install and set up (IMO anyway) it is a fraught for those less experienced and there are numerous threads here where people have hit issues. I have ideas of how to improve that and make installing alternate PHP versions super easy for TurnKey users, but I just haven't had enough spare cycles to put it all together... One day I hope.

However the next major Debian release (12/Bookworm - the basis of the upcoming v18.0) has PHP v8.2. So whilst I mentioned (in the other thread) that technically we don't support Debian "in place" OS upgrades, there is nothing wrong with doing it that way and in this particular case, I think that an OS upgrade is probably actually a quicker/easier pathway to get you up to date.

So to do a full OS upgrade, just follow the Debian docs. Although please note that they are fairly verbose and cover things that I wouldn't bother with myself. So I suggest that you have a read through that, but then have a google for some more straight forward, step-by-step instruction. If you google something like "how to upgrade debian bullseye server to bookworm" you should get oodles of good tutorials and perhaps even a video or 2.

Also, if you have any questions regarding details, please ask and I'll do my best to answer (or at least point you in the right direction).

As for the process, definitely take a snapshot of your server (i.e. a Proxmox backup) before you start doing anything to your current server. Personally I use TKLBAM too (for remote encrypted backups), but that's not neccessary and is somewhat redundant in this scenario.

I suggest after taking a Proxmox backup, that you restore that to a new instance and do your upgrade there first. Take notes as you go and probably a few progress snapshots/backups along the way. That way if something goes pear shaped, you can roll back and try again using your notes and/or snapshots to get you quickly back to where you were. Once you're good, you can then re-run the steps on your original server. It also means that there is no interruption to your existing server while you work on this - meaning less stress and pressure to "get it done" (which IMO always makes life a bit nicer). Another thought, if you haven't changed any of your Nextcloud data while you do the upgrade, you could probably just swap the IP addresses and retire the old server in favor of the new server once it's ready? (Rather than applying your upgrade notes to your original server)

Once you have updated the OS, then you can upgrade Nextcloud itself. Either via the web UI or via the CLI. In case you weren't aware, we provide a "occ" wrapper script (to run 'occ' commands as the webserver user, so as to not upset permissions) - it's 'turnkey-occ' (it looks like it's been there for a while, so you should already have that in v17.1).

Obviously that's not step-by-step and I'm sure you'll have more questions, but hopefully that's a good start. If you go this way, it'd be awesome if you were willing top publish your notes as I'm sure it will help others.

Data migration

The other option is to migrate your data to a new server. I recommend using TKLBAM but there's no reason it couldn't be done manually if you're so inclined (either completely manually, or using TKLBAM in 'solo' mode). Please note that by default TKLBAM requires a Hub account (which in turn requires an AWS account). When you sign up to the Hub, you will start a free 14 day trial. If you don't intend to run Cloud servers, be sure to cancel the Cloud server free trial and if you don't intend to use TKLBAM beyond this transition, you can also cancel the TKLBAM trial (the Free Backup plan supports one server backup so would be adequate for this process - assuming you have no other backups).

Using TKLBAM is the path that we generally recommend and in a perfect world, it "just works" (as noted in the docs, it should always "just work" when restoring to the same server, same turnkey version - not so much when transferring between versions, especially major versions). Unfortunately we don't quite live in that world. So I generally recommend using a "staged" TKLBAM resore (rather than just a full restore).

That would look something like this:

# existing server
tklbam-backup --full-backup now
# then on new server
mkdir /tklbam-dump
tklbam-restore BACKUP_ID --raw-download=/tklbam-dump
# where BACKUP_ID is the back ID number - probably '1' on a new Hub account
tklbam-restore /tklbam-dump --skip packages --limits="/etc/apache2 /var/www mysql:nextcloud"

Note that will only restore your Apache config, Nextcloud data and the Nextcloud DB - but Nextcloud still won't work yet...

To update the Nextcloud DB password (so Nextcloud can access it's own DB) you could probably just run the inithook: /usr/lib/inithooks/firstboot.d but it does some other things that I'm not 100% sure are safe on an existing system. So just in case that breaks things, here's the commands to just reset the Nextcloud DB pass:

PASSWORD=$(mcookie)
sed -i "s|dbpassword.*|dbpassword' => '$PASSWORD',|" /var/www/nextcloud/config/config.php
/usr/lib/inithooks/bin/mysqlconf.py --user=nextcloud --pass="$PASSWORD"

That should get things pretty much working as they are. Although you may need to reapply any PHP config tweaks you previously applied (which you'd need to do anyway - PHP config is version specific, so changing to a new version of PHP means you need to reconfigure any manually applied PHP tweaks.

If you have other custom config that you know you want, you can extend the '--limits="..."'. E.g. if you wanted to include 'somedir' from root's home (i.e. /root/somedir), the tklbam restore line will look like:

tklbam-restore /tklbam-dump --skip packages --limits="/root/somedir /etc/apache2 /var/www mysql:nextcloud"

Also, your full backup will be found in /tklbam-dump, relative to /. E.g. the previously mentioned /root/somedir directory will be found at /tklbam-dump/root/somedir. SO you can manually copy out stuff from there too if you wish. However, please be aware that if you move files manually, it may mess with permissions. TKLBAM should restore permissions to original. So generally you're better off using TKLBAM directly (e.g. to just restore /root/somedir: 'tklbam-restore /tklbam-dump --skip packages --limits="/root/somedir"'). Note that TKLBAM does have a "tklbam-restore-rollback" command, which as the name suggests, allows you to rollback a restore. Please note though, that it only supports one rollback. So if you run multiple 'tklbam-restore' commands, 'tklbam-restore-rollback' will only allow the rollback of the most recent restore.

If going this latter path, then once you are happy, then you can change the IP address of your old server and update the IP of the new one.

Final words

As I noted at the top of this post, I'm currently working on v18.0 release, which will ship with PHP 8.2 by default. So if you like the sound of the second option better, and your need to update isn't urgent, then perhaps it's worth waiting a few weeks and go straight to v18.0?

Also I just had one more thought (it should probably go above somewhere, but I need to get back to working on v18.0). If you wish to do the upgrade path, then things are pretty much in place ready to go (e.g. the TurnKey bookworm repo exists and has latest packages ready to go). Although you will need to manually set up the new Bookworm TurnKey apt repo key(s). That isn't as easy as it should be but here's a quick and dirty run through:

# UPDATE - I just quickly tested this and apt can now use asc files, so we can remove one line
URL=https://raw.githubusercontent.com/turnkeylinux/common/18.x-dev/overlays/bootstrap_apt
KEYPATH=usr/share/keyrings
for repo in main security testing; do
    key="tkl-bookworm-$repo.asc"
    keyring="/$KEYPATH/$keyfile.gpg"
    wget -O /$KEYPATH/$key.asc $URL/$KEYPATH/$key.asc
done

You'll also need to update the keypath in the apt sources. Although if you use sed to automatically update the sources.list files, then that will take care of it for you. I.e.:

sed -i.bak "s|bullseye|bookworm|" /etc/apt/sources.list.d//{sources.list,security.sources.list,turnkey-testing.list}

As per my update to the code to download the keys, you'll also need to change the key path to use the .asc file extension, rather than .gpg. So update your sources list like this:

sed -i.bak2 "\|turnkeylinux.org| s|gpg|asc|" /etc/apt/sources.list.d/{sources.list,security.sources.list,turnkey-testing.list}

There's probably tons of other titbits of info in my head, but I can't think of anymore right now. Good luck with it all and please don't hesitate to post back with questions. I'll try to respond within a reasonable timeframe.

Robot Unsalted's picture

Hi Jeremy,

Once again, thanks for taking time out of your day and replying with such detail.

I appreciate you pointing out that tklbam has a local option, which is really interesting and worth exploring.

To reply to your earlier comment about upgrade PHP, which frankly was my main reason to look to update. I did attempt to upgrade the PHP using a variety of different posts on this forum, troubleshooting, and personally coming up with nothing.

In all honesty, I will likely wait until version 18, which I am excited to hear will have an easier migrate process. Turnkey Linux has been quite true to its name and I am glad that the upgrade process will be joining that.

Add new comment