(2022-08-25) Move webseitz to DigitalOcean

I moved FluxGarden to DigitalOcean, but nothing else at LiNode. So webseitz is still running from there...

Where do I want to end up?

same code as FluxGarden - if I'm experimenting with something I can use a Feature Toggle
share database with FluxGarden

Later

then generate WikiGraph footers directly in FluxGarden code, dumping remove WikiGraph call
(ideally/later) have webseitz start using ssl, leave non-ssl running (pick SSL to be canonical, and let people's browsers nudge them to it)
then bring WikiGraph code over (have to move from web.py to flask but that should be easy) - support with SSL and non-SSL depending on the preference of the site who's embedding it

Where are things now?

whereis nginx -> nginx: /usr/sbin/nginx /usr/lib/nginx /etc/nginx /usr/share/nginx /usr/share/man/man8/nginx.8.gz
nginx -v -> nginx version: nginx/1.18.0 (Ubuntu)
/etc/nginx/nginx.conf is fairly generic, meat is in /etc/nginx/sites-enabled/flux.garden.conf
- some interesting ranting against sites-enabled, but I'm not going to change it (I appear to have the same flux.garden.conf file in both sites-enabled and sites-available, rather than a symlink. Also, I've never used the scripts like nginx_ensite
I have nothing inside /etc/nginx/conf.d
other key config files seems to be /etc/systemd/system/wikiweb.service and /home/bill/wikiweb/wikiweb_uwsgi.ini

So the big/first question for me is: do I even need a separate service running to handle the new domain, if it's the same code being used?

friend recommends: one nginx instance, two redirect rules to two different proxies, two different flask daemons, two different systemd scripts

Feb26'2023

the existing service is called wikiweb, I'm going to call the new one fluxent
:/etc/systemd/system$ sudo cp wikiweb.service fluxent.service then edit
~/wikiweb$ cp wikiweb_uwsgi.ini fluxent_uwsgi.ini then edit
there's a ~/uwsgi_sockets/wikiweb.socket which seems to symlink to itself? Do I manually create this, or does it get created automatically if I define that elsewhere (I think in the nginx config)? We'll move on and see....
/etc/nginx/sites-available$ cp flux.garden.conf fluxent.com.conf then edit (not supporting SSL this round) (and copy file to ../sites-enabled/
sudo systemctl restart fluxent....
ps awux | grep fluxent → see bunch of processes
need to edit my laptop hosts to point there to try hitting it → nope, redirects to flux.garden
probably need to restart nginx → service nginx restart → wants a password
sudo nginx -s reload → nope redirects to flux.garden

Feb27

set debug-logging for nginx - do I have SSL turned on for fluxent.com? Doesn't look like it...
ah, the redirect to flux.garden is happening inside the Flask code! Yep, confirmed in the flask/uwsgi log.
ugh, SERVER_NAME is hard-coded in .env - will there be an issue if I make it more dynamic? Do I need a separate flask install for each domain?
also note: running

Python 3.8.5
Flask 1.1.2
Werkzeug 1.0.1

there are ways to support multiple domains with a single Flask instance
- host-matching: https://ron.sh/two-domains-one-flask/
  - see also https://github.com/pallets/flask/issues/2813
- a different way: https://keremkoseoglu.wordpress.com/2021/08/07/hosting-multiple-domains-on-python-flask/
- my friend agrees host-matching is reasonable for a small number of sites
  - when I get to migrating other sites, I'll probably have separate process, even for WikiGraph which might share the same db (though maybe not, since I'll be using it locally for fluxent.com and flux.garden)

Apr23

edit my hosts file - it already has a few .flux2.garden subdomains (including webseitz.flux2.garden, now adding webseitz.fluxent2.com
launch my local flask
- http://webseitz.flux2.garden:5000 gives me http://webseitz.flux2.garden:5000/wiki/FrontPage nicely
- http://webseitz.fluxent2.com:5000 gives me error from space = Space.query.filter(Space.owner_id == owner.id).filter(Space.privacy_type == 'wiki').first()
- http://webseitz.fluxent2.com:5000/wiki redirects me to http://%3Cinvalid%3E.flux2.garden:5000/wiki/
check my db record: users.id=1 -> subdomain='webseitz' so the issue is the domain mapping, not the subdomain.
modify /app/__init__.py as app = Flask(__name__, host_matching=True) -> nope have to add the static_host bit...
I have no app.run() calls
now http://webseitz.flux2.garden:5000/wiki gives 404-not-found
- no SQL query displayed in the terminal window - sigh it's been a long time since I touched this stuff....
I commented out SERVER_NAME in .env file, and associated line in config.py -> still getting the 404 (didn't expect it to change, but figured I'd blank it out now)
I'm going to focus on page BillSeitz http://webseitz.flux2.garden:5000/wiki/BillSeitz - which also gives 404 right now
check my flask version:

flask --version
Python 3.8.5
Flask 1.1.2
Werkzeug 1.0.1

Apr26

I realize I've never had subdomain_matching=True set, so I add it
decorator line is now like @app.route('/<path>/<page_url>', subdomain='<subdomain>', host="flux2.garden:5000")
→ TypeError: page() missing 1 required positional argument: 'subdomain' so I guess the subdomain parsing isn't working
I asked ChatGPT about host_matching, and it gave me some examples!
@app.route('/<path>/<page_url>', subdomain='*', host='flux2.garden:5000') → now getting 404 again!
@app.route('/<path>/<page_url>', subdomain='webseitz', host='flux2.garden:5000') → also 404!
I have a logger-line right inside the page() function, and it doesn't get called, so the issue is in the route matching, not something inside the function.
ChatGPT also suggested adding host_matching to the route decorator → @app.route('/<path>/<page_url>', subdomain='webseitz', host='flux2.garden:5000', host_matching=True) → nope TypeError: __init__() got an unexpected keyword argument 'host_matching'
try getting rid of subdomain as separate argument

@app.route('/<path>/<page_url>', host='http://webseitz.flux2.garden:5000')
def page(page_url, path, host):

→ nope 404

May01: decide i don't trust my local repo

download from gitlab
copy .env and .flaskenv from local-old folder into fresh folder; flask run works
hit http://webseitz.flux2.garden:5000/ → KeyError: SERVER_NAME (which makes sense, since I deleted it from .env as part of host_matching)
same error → oh yeah have to remove SERVER_NAME line from config.py, too
now I get the 404 again

Aug27 reconsidering my options for moving forward

Fresh venv, fresh flask, copy app, launch, make multi host
Upgrade DO. Install separate code copy, live with it.
yep, decide the latter, it's making more sense to me to have 2 similar-but-independent code bases, esp since I want to noodle on some new ideas... but I will aim to run them both from the same db (not just server, but specific db)
make new fluxent directory. git clone into it (using https) -> /fluxent/wikiweb/
that doesn't copy .env file, so do it by hand
- edit SERVER_NAME and SERVER_PROTOCOL (just http not https)
mv fluxent_uwsgi.ini ~/fluxent/wikiweb/.
- edit paths in file
edit /etc/nginx/sites-available/fluxent.com.conf (and copy to sites-enabled)

Aug30

do sudo nginx -s reload -> nginx: [warn] conflicting server name "fluxent.com" on 0.0.0.0:80, ignored
sniffing around, notice I have both fluxent_uwsgi.ini and wikiweb_uwsgi.ini in ~/fluxent/wikiweb (and the latter file is also over in its proper directly). So rm wikiweb_uwsgi.ini in the fluxent tree. Nope, doesn't change anything
maybe I need more than reload, do sudo systemctl restart nginx - no change
if I do ps awux | grep fluxent I only see 1 process, whereas if I do it for wikiweb I see a whole bunch.
look inside /var/log/nginx/error.log notice connect() to unix:///home/bill/uwsgi_sockets/fluxent.socket failed (2: No such file or directory)
go into ~/uwsgi_sockets find wikiweb.socket, no fluxent.socket
/etc/nginx/sites-available/fluxent.com.conf contains uwsgi_pass unix:///home/bill/uwsgi_sockets/fluxent.socket;
maybe chown issues? (though everything is consistent across the old and new apps)
- the app-code folders all have bill:bill
- ~/uwsgi_sockets are bill:bill
- /var/log/nginx/* are all www-data:adm
- /var/log/uwsgi/* are all bill:bill (and they're all empty)
- /etc/nginx/* are all root:root

Sept04

derp I haven't been manually killing my uwsgi services, so I think restarting nginx tries to trigger things running alongside the old processes...
sudo pkill -f uwsgi -9 -> bunch of uwsgi processes immediately respawned
(noticing that up at Feb26 I had a "bunch" of fluxent processes running but now doing ps awux | grep fluxent just gives me one
sudo systemctl restart fluxent -> Job for fluxent.service failed because the control process exited with error code. See "systemctl status fluxent.service" and "journalctl -xe" for details.
journalctl -xe -> realpath() of /home/bill/wikiweb/fluxent_uwsgi.ini failed: No such file or directory [core/utils.c line 3662]... fluxent.service: Executing: /bin/sh -c 'exec /usr/local/bin/uwsgi --ini /home/bill/wikiweb/fluxent_uwsgi.ini'
ah, /etc/systemd/system/fluxent.service needs to be edited
edit, restart -> same error msg
sudo grep -Rn '/' -e '/bill/wikiweb/fluxent'
derp had the wrong path-order inside fluxent.service
kill/restart -> same error
- except journalctl -xe doesn't give me anything fresh
systemctl status fluxent.service ->

fluxent.service - fluxent uWSGI Dev
     Loaded: loaded (/etc/systemd/system/fluxent.service; disabled; vendor preset: enabled)
     Active: failed (Result: resources) since Mon 2023-09-04 18:32:54 UTC; 1min 26s ago

found post that suggested checking syslog (because should give detail behind the Result: resources bit)

Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Installed new job fluxent.service/restart as 1004362
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Enqueued job fluxent.service/restart as 1004362
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Scheduled restart job, restart counter is at 5.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202270 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202271 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1 interface=org.freedesktop.systemd1.Manager member=JobNew cookie=202272 reply_cookie=0 signature=uos error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202273 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/fluxent_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=202274 reply_cookie=0 signature=sa{sv}as error-name=n/a error-message=n/a
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Changed auto-restart -> dead
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Job 1004362 fluxent.service/restart finished, result=done
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: Stopped fluxent uWSGI Dev.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Converting job fluxent.service/restart -> fluxent.service/start
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Start request repeated too quickly.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Failed with result 'resources'.
Sep  4 18:32:54 ubuntu-s-2vcpu-4gb-nyc3-01 systemd[1]: fluxent.service: Service restart not allowed.

tried manual command-line /bin/sh -c 'exec /usr/local/bin/uwsgi --ini /home/bill/fluxent/wikiweb/fluxent_uwsgi.ini' -> no error, and bunch of fluxent processes running. And can his the URL that was failing (with 502) before!
- and now getting non-empty /var/log/uwsgi/wikiweb.log and fluxent.log
if I ctl-c the uwsgi window, all the fluxent processes go away, and I start getting the 502 again.
- and FYI the tail of fluxent.log is

SIGINT/SIGQUIT received...killing workers...
worker 1 buried after 1 seconds
worker 2 buried after 1 seconds
worker 3 buried after 1 seconds
worker 4 buried after 1 seconds
worker 5 buried after 1 seconds
worker 6 buried after 1 seconds
worker 7 buried after 1 seconds
worker 8 buried after 1 seconds
goodbye to uWSGI.
VACUUM: pidfile removed.
VACUUM: unix socket /home/bill/uwsgi_sockets/fluxent.socket removed.

Oct08 - friend is creating new infrastructure for me, involving Docker in some way... so while waiting I'm going to focus on getting my local code working again, ripping out any of the multi-domain stuff... I'm going to focus on my wikiweb-main folder, checking the config bits from my wikiweb_20210425 snapshot

ah, it was the __init__.py, ripping the new junk of out app = Flask(__name__)

Edited: 2023-10-08 16:12:12.011049 | Tweet this! | Search Twitter for discussion

Bill Seitz