Endgame Viable

I want to make a plan to migrate a gotosocial instance and postgres server installed on my local Unraid server to an Internet virtual server of some kind. Can you give me some options? Is it even possible?

1 - I would like to use a low-cost, self-managed VPS, I may already have one actually 2 - it uses local storage I think, 3 - the script connected to the postgres database directly

Will I be able to migrate the existing gts.endgameviable.com domain to the new server so that federation continues to work seamlessly

I have a VPS plan at Vultr with these specs, will that be enough to run a low-traffic gotosocial and postgres containers? AMD High Performance
1 vCPU, 1024 MB RAM, 25 GB NVMe, 2.00 TB Transfer

services:
  postgres:
    image: postgres:15-alpine
    container_name: gts-postgres
    restart: always
    environment:
      POSTGRES_DB: gotosocial
      POSTGRES_USER: gotosocial
      POSTGRES_PASSWORD: "YOUR_SECURE_PASSWORD" # Match or update from Unraid
    volumes:
      - ./postgres_data:/var/lib/postgresql/data
    # Note: We do NOT expose port 5432 to the public internet for security.

  gotosocial:
    image: superseriousbusiness/gotosocial:latest
    container_name: gotosocial
    restart: always
    environment:
      GTS_HOST: gts.endgameviable.com
      GTS_DB_TYPE: postgres
      GTS_DB_ADDRESS: postgres
      GTS_DB_PORT: 5432
      GTS_DB_USER: gotosocial
      GTS_DB_PASSWORD: "YOUR_SECURE_PASSWORD"
      GTS_DB_DATABASE: gotosocial
      GTS_STORAGE_BACKEND: local
      GTS_STORAGE_LOCAL_DIR: /gotosocial/storage
      GTS_ACCOUNTS_REGISTRATION_OPEN: "false"
      GTS_PORT: 8080
    volumes:
      - ./gts_storage:/gotosocial/storage
    depends_on:
      - postgres

  caddy:
    image: caddy:2-alpine
    container_name: caddy-proxy
    restart: always
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      - gotosocial

volumes:
  caddy_data:
  caddy_config:

gts.endgameviable.com {
    reverse_proxy gotosocial:8080
}

import urllib.request
import json

# Your GoToSocial account ID
account_id = "redacted"
api_url = f"https://gts.endgameviable.com/api/v1/accounts/{account_id}/statuses?limit=40"

# Fetch statuses from the public API
req = urllib.request.Request(api_url, headers={'User-Agent': 'GTS-Archiver'})
try:
    with urllib.request.urlopen(req) as response:
        statuses = json.loads(response.read().decode())
except Exception as e:
    print(f"Error fetching from API: {e}")
    sys.exit(1)

# Filter and format statuses in-memory (replacing the SQL conditions)
result_rows = []
for status in statuses:
    created_at_dt = datetime.strptime(status["created_at"], "%Y-%m-%dT%H:%M:%S.%fZ")
    
    # Apply date filters (start/end)
    if not (yesterday <= created_at_dt < today):
        continue
        
    # Apply content filters
    if "https://gts.endgameviable.com/tags/notification" in status["content"]:
        continue
        
    # Map API response fields to match the rest of your script
    result_rows.append({
        "id": status["id"],
        "in_reply_to_id": status["in_reply_to_id"],
        "url": status["url"],
        "visibility": status["visibility"],
        "content_warning": status["spoiler_text"],
        "content": status["content"],
        "created_at": status["created_at"],
        "dayname": created_at_dt.strftime("%A"),
        "day": created_at_dt.strftime("%m/%d"),
        "hour": created_at_dt.strftime("%H:%M")
    })

I've created a new directory gts-orchestration and would like to use that directory to setup a local test instance of the gotosocial stack

Is there a way to export Unraid docker container vars or do I have to type them by hand from the UI?

Here are my existing gotosocial vars, update the plan to use the same vars

oot@Tower:~# docker inspect --format='{{range .Config.Env}}{{println .}}{{end}}' gotosocial
GTS_ACCOUNTS_REGISTRATION_OPEN=false
GTS_STATUSES_MAX_CHARS=2000
GTS_LETSENCRYPT_EMAIL_ADDRESS=[REDACTED_EMAIL]
GTS_DB_TYPE=postgres
HOST_HOSTNAME=Tower
GTS_HOST=gts.endgameviable.com
GTS_DB_USER=gotosocial
GTS_DB_PASSWORD=[REDACTED_PASSWORD]
GTS_LANDING_PAGE_USER=ultrviolet
GTS_INSTANCE_EXPOSE_PUBLIC_TIMELINE=true
HOST_OS=Unraid
HOST_CONTAINERNAME=gotosocial
GTS_DB_ADDRESS=[REDACTED_IP]
GTS_LETSENCRYPT_ENABLED=false
GTS_LOG_LEVEL=debug
TZ=America/New_York
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Update the plan to configure env vars for docker compose for postgres password and GTS_HOST

Update the plan so the postgres password is only in the .env file

Update the plan to remove "ultrviolet" from docker-compose.yml and .env.sample

Can you update the plan to add Makefile targets to add and remove gts.endgameviable.com to /etc/hosts - is that possible?

I have docker desktop installed but I get this

tkrehbiel@MacBook-Pro gts-orchestration % make up
docker compose up -d
make: docker: No such file or directory
make: *** [up] Error 1

It seems like it's already installed... it doesn't let me apply

Now I get this

tkrehbiel@MacBook-Pro gts-orchestration % make up
/usr/local/bin/docker compose up -d
[+] up 0/3
 ⠋ Image postgres:15-alpine                     Pulling                                                    0.0s
 ⠋ Image superseriousbusiness/gotosocial:latest Pulling                                                    0.0s
 ⠋ Image caddy:2-alpine                         Pulling                                                    0.0s
error getting credentials - err: exec: "docker-credential-desktop": executable file not found in $PATH, out: ``
make: *** [up] Error 1

It started up, now what?

Can I open anything in a browser without setting up gts.endagmeviable.com?

For now I don't want to restore a database, I just want to see if it runs before I work on migrating data

Okay it works, now I want to start making a plan to migrate data from the Unraid postgres database. How do I export data from postgres?

I've never ssh'ed to the unraid server, is it just "ssh tower.local"?

tkrehbiel@MacBook-Pro gts-orchestration % ssh root@tower.local
ssh: connect to host tower.local port 22: Connection refused

I don't see appdata in the list of shares.. it's got a warning icon next to it in the unraid UI

The postgres backup only has 54 bytes, I don't think it worked

pg_dump: error: invalid output format "-c" specified

I've imported the backup files into the project dir, make a plan to import them into my local test setup

I got these warnings

root@Tower:~# docker exec -i postgresql15 pg_dump -U gotosocial -F c -d gotosocial > /mnt/user/LocalBackups/gotosocial_backup.dumpWARNING:  database "gotosocial" has a collation version mismatch
DETAIL:  The database was created using collation version 2.36, but the operating system provides version 2.41.
HINT:  Rebuild all objects in this database that use the default collation and run ALTER DATABASE gotosocial REFRESH COLLATION VERSION, or build PostgreSQL with the right library version.

I updated the dump file, continue the plan

The site said this, how do I see logs

An error occured:
Internal Server Error
Request ID: 000037ya5a58p0000010

I see this warning at the top of the gts page

Warning! It looks like either trusted-proxies is not set correctly in this instance's configuration, or the upstream reverse proxy is not passing an appropriate X-Forwarded-For or X-Real-IP header.

This may cause rate-limiting issues and, by extension, federation issues.

If you are the instance admin, you may be able to fix this by adding [REDACTED_IP]/16 to your trusted-proxies.

For more information and other possible fixes for this issue, see the documentation.

So that was an issue in the local environment only, and wouldn't have happened in a live environment?

POSTGRES_PASSWORD=[REDACTED_PASSWORD]
GTS_HOST=gts.endgameviable.com
GTS_STATUSES_MAX_CHARS=2000
GTS_LANDING_PAGE_USER=ultrviolet
GTS_INSTANCE_EXPOSE_PUBLIC_TIMELINE=true
GTS_LOG_LEVEL=debug
TZ=America/New_York

# Stop containers and delete local docker volumes
make clean

make up

# Extract the tarball (creates a "storage" directory)
tar -xzvf gotosocial_storage.tar.gz

# Rename the folder to match the docker volume mount path
mv storage gts_storage

make restore-db DUMP=gotosocial_backup.dump

make hosts-add

Now that I've tested locally, how can I install this setup on my Vultr VPS server?

Make a plan, don't change things

Vult instance os is Ubuntu 22.04 x64

I currently have gts.endgameviable.com pointed to a CloudFront distribution... should I continue to use the CloudFront distribution and point it to the Vultr server? Or is that a waste?

Is there any risk of breaking federation during the time period when I switch gts.endgameviable.com but haven't yet started the gotosocial server?

If a server is declared dead in federation, and it comes back months later, will federation resume or is it dead forever?

How do I verify that the domain name is pointing to the Vultr server if there's no web server on the VPS server

tkrehbiel@MacBook-Pro gts-orchestration % tar -czvf gts-deploy.tar.gz docker-compose.prod.yml Caddyfile.prod .e
nv gotosocial_backup.dump gotosocial_storage.tar.gz
tar: docker-compose.prod.yml: Cannot stat: No such file or directory
tar: Caddyfile.prod: Cannot stat: No such file or directory
a .env
a gotosocial_backup.dump
a gotosocial_storage.tar.gz
tar: Error exit delayed from previous errors.

What next?

I've just updated the OS on the VPS to ubuntu 24.04

As part of the plan, create a shell script to run all the necessary ubuntu setup from a fresh os install, and make it idempotent

The setup script should also setup starting docker compose on rebooting

Add a make target to package files for transfer

I packaged the files and scp'ed them to the VPS, now what?

Is this an issue?

root@development:~# tar -xzf /tmp/gts-deploy.tar.gz -C ~/gts-orchestration
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'
tar: Ignoring unknown extended header keyword 'LIBARCHIVE.xattr.com.apple.provenance'

I got a ton of errors restoring the pg backup

root@development:~/gts-orchestration# docker compose exec -T postgres pg_restore -U gotosocial -d gotosocial --clean < gotosocial_backup.dump
pg_restore: error: could not execute query: ERROR:  index "web_push_subscriptions_account_id_idx" does not exist
(snip)
Command was: DROP TABLE public.account_notes;
pg_restore: warning: errors ignored on restore: 256

I don't see a page loading at gts.endgameviable.com yet

root@development:~/gts-orchestration# docker compose ps
NAME           IMAGE                                    COMMAND                  SERVICE      CREATED              STATUS              PORTS
caddy          caddy:2-alpine                           "caddy run --config …"   caddy        About a minute ago   Up About a minute   0.0.0.0:80->80/tcp, [::]:80->80/tcp, 0.0.0.0:443->443/tcp, [::]:443->443/tcp, 443/udp, 2019/tcp
gotosocial     superseriousbusiness/gotosocial:latest   "/gotosocial/gotosoc…"   gotosocial   About a minute ago   Up About a minute   
gts-postgres   postgres:15-alpine                       "docker-entrypoint.s…"   postgres     About a minute ago   Up About a minute   5432/tcp
root@development:~/gts-orchestration# docker compose logs caddy
caddy  | {"level":"info","ts":1785877892.8641872,"msg":"maxprocs: Leaving GOMAXPROCS=1: CPU quota undefined"}
(snip)
caddy  | {"level":"info","ts":1785877898.657089,"logger":"tls.obtain","msg":"releasing lock","identifier":"gts.endgameviable.com"}

The gotosocial log is enormous but I don't see any errors

root@development:~/gts-orchestration# curl -Iv https://gts.endgameviable.com
* Host gts.endgameviable.com:443 was resolved.
(snip)
< 
* Connection #0 to host gts.endgameviable.com left intact

https://gts.endgameviable.com/@ultrviolet seems to work though

Chrome is saying my connection to gts.endgamevaible.com is not secure