(2012-10-16) Setting Up Hosting For Simplest Thing

Pretty much done with v1 of Simplest Thing, time to put up on a Hosted Server.

PaaS vendors are too expensive when you're a Micropreneur putting up a paid (thus low-load) service.

Going with Linode, largely because friend offered to help there. Otherwise probably would have gone with Web Faction. Just $20/mo.

(Have built with WebPy - WebpyForSimplest Thing.)

Looks like will end up running with NginX, Postgre[[SQL]], and WSGI (uWSGI - UWsgi).

  • note that page is 18mo old, so will pick newer versions of things to install.
    • Ubuntu 11.10
    • uwsgi-1.2.6
    • nginx-1.3.5

When get to point of launching NginX, get Starting nginx: nginx: [emerg] getpwnam("nginx") failed grrr how much do I hate it when recipes don't work.

  • had to play some games with setting owner in nginx.conf and doing chown/chmod stuff. Done Oct17.

Config NginX virtual server.

  • mostly taking background info from here
    • except looks like files will be served from /srv/www/ not /var/www/
  • just add a server block inside nginx.conf instead of playing with /sites-available/
    • and making the /static directory a sub of /app/ where my code will be, since that's where WebPy tends to put things.
  • /etc/init.d/nginx restart server - comes up after I do /mkdir/ for the /logs/
  • but hitting the IP# doesn't give me anything. Is that because I need to make requests by the hostnames used in the server block?
    • made local hosts entry to point to server by name. Still not connecting.
    • played chown/chmod games in /srv/www/ - no change
    • noting that I've been playing with 2 different nginx.conf files - one in /opt/nginx/ and the other in /opt/nginx-1.3.5/ - and today's edits were only in the former.
    • find /opt/nginx/logs/error.log saying 2012/10/17 09:10:04 [emerg] 21887#0: open() "/srv/www/simplest-thing.com/logs/access.log" failed (2: No such file or directory). So copy the 2 log files from /opt... to /srv... (at 10:18am). Restart server again. Same result. No additions to either set of log files.
    • hmm, could problem be that I added that new server block to nginx.conf but left in already-existing block that also uses port-80 (for localhost)?
      • tweak that original server block to listen on 81. Restart. Still won't connect, still nothing added to logs.
      • do chown/chmod on logs in both directories. Restart. Still won't connect, still nothing added to logs.
    • grr, should have checked that server was actually running before I started this step.
    • my friend isn't available, what can I try on my own?
      • rip out the Server Block I added? No, actually start back from default.
      • play with the fact that there's 2 config files - maybe rename 1 to get it ignored?
        • renamed /opt/nginx-1.3.5/conf/nginx.conf to nginx.conf.seitz. Restart, no change.
      • in /opt/nginx/conf rename my latest nginx.conf to nginx.conf.seitz and then copy nginx.conf.default to nginx.conf
      • do chown/chmod on that file again. No change.
      • chang user line inside nginx.conf - try user name my friend had said, also tried nginx. Neither worked. (And still nothing in any nginx logs, nor anything useful in /var/log/.
    • my friend finds simple fix - just change line in nginx.conf that says to listen on localhost to listen on the specific IP (Oct22)
  • I edit nginx.conf to add server block per instruction page. Restart. Now get 502 Bad Gateway response.
    • the /opt/nginx/logs/access.log was just updated with line 76.245.240.208 - - [22/Oct/2012:16:23:43 -0500] "-" 400 0 "-" "-"
    • the /srv..../access.log was updated with line {{{ 76.245.240.208 - - [22/Oct/2012:16:23:33 -0500] "GET / HTTP/1.1" 502 574 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) Apple Web Kit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.26 Safari/537.11" 76.245.240.208 - - [22/Oct/2012:16:23:33 -0500] "GET /favicon.ico HTTP/1.1" 502 574 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_7) Apple Web Kit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.26 Safari/537.11" }}}
    • the /srv/.../error.log was updated with {{{ 2012/10/22 16:23:33 [error] 25135#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 76.245.240.208, server: www.simplest-thing.com, request: "GET / HTTP/1.1", upstream: "uwsgi://127.0.0.1:9001", host: "www.simplest-thing.com" 2012/10/22 16:23:33 [error] 25135#0: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 76.245.240.208, server: www.simplest-thing.com, request: "GET /favicon.ico HTTP/1.1", upstream: "uwsgi://127.0.0.1:9001", host: "www.simplest-thing.com" }}}
    • edit nginx.conf file again to change the original server blog to listen on 81 instead of 80. Restart. No change.
    • do mkdir /srv/www/simplest-thing.com/app/static/ then create index.html inside it. Restart. Try to hit http://www.simplest-thing.com/static/index.html get 404 Not Found error. Log looks similar, though notice some lines like 2012/10/22 16:41:49 [error] 25233#0: *1 open() "/srv/www/simplest-thing.com/app/static/static" failed (2: No such file or directory), client: 76.245.240.208, server: www.simplest-thing.com, request: "GET /static HTTP/1.1", host: "www.simplest-thing.com" mean I have static being mapped twice, or I'm not supposed to include static in the URL, or something.
    • I also try hitting http://www.simplest-thing.com/wsgi_configuration_module.py for kicks, but get 502. Should a request for / be automatically calling that to give me the Hello World response?
    • kicking it back to my friend.... who says...
    • derp you need to launch a uwsgi daemon that runs on port 9001. See her comments below. Also see this page that I just found.

Install Postgre[[SQL]] - apt-get install postgresql python-psycopg2

Install WebPy - apt-get install python-webpy

Use scp to copy all my app stuff from my Lap Top up to server - in /srv/.../app/.

  • includes .sql dump

Next: set up Postgre[[SQL]] database and user so can import from dump.

  • createuser webpy
  • createdb mf
  • hrm, I should probably switch over on my Lap Top, to avoid having to do this repeatedly. (And to avoid environment-specific code changes.)
  • ah, can't use pgdump to import data until convert that dump file from SQLite style.
    • start with these notes to make edits to dump file
      • make ID fields serial primary key not just serial
    • delete sqlite_sequence table and data.
    • re-arrange tables so that "base" tables come before detail-tables that reference them
    • had a table with lots of records having an empty price value that was set like empty-string instead of null or 0. Changed them all to 0.
    • had a field that was supposed to be just varchar(8) but had many values that were longer than that! Changed field spec.
    • had a text field set to varchar(1024) but max length of data I was importing was 1070. Changed all such fields to type text.
    • success!
  • hrm, psql stores each serial/sequence in its own table. So I have a number of tables created, but each has all its values set to 1, like {{{ sequence_name | last_value | start_value | increment_by | max_value |min_value | cache_value | log_cnt | is_cycled | is_called ---------------+------------+-------------+--------------+---------------------+----------+-------------+---------+-----------+----------- notes_id_seq | 1 | 1 | 1 | 9223372036854775807 | 1 | 1 | 0 | f | f }}}
    • list all the sequences with select relname from pg_class where relkind = 'S'; (Oct14)
    • for each case, do SELECT setval('your_table_id_seq', (SELECT MAX(id) FROM your_table));

Edit my WebPy code.py for new db settings. (Will have to make config system later.) Use user='webpy' with no password.

  • test this user theory by doing commandline psql user=webpy mf
    • get psql: FATAL: Peer authentication failed for user "webpy"
    • re-created user webpy with a password
    • edit /etc/postgresql/9.1/main/pg_hba.conf for its local connection use md5
    • restart psql sudo service postgresql restart
    • test connect with command line - get prompted for password
    • change WebPy code.py to include password param.

Get uwsgi pointing to my code.py

  • edit /etc/default/uwsgi (though suspect that doesn't matter)
  • edit start_uwsgi_app.sh created by my friend
  • use pgrep uwsgi to find the running uwsgi processes, then kill each
  • do bash start_uwsgi_app.sh to start new uwsgi
  • hit url with browser, get uWSGI Error | Python application not found
  • this doesn't surprise me, as I had already seen some instructions for using web.py with uwsgi that require more tweaks to code. But figured I try anyway.
  • add application = app.wsgifunc() to the very end of the code.py (even outside the __name__ == __main__ test)
  • add sys.path.append('/srv/www/simplest-thing.com/app') up at the top of code.py
  • kill/restart uwsgi
  • hit URL again
    • redirects me to my static splash page as it's supposed to! (since I'm not logged in)
    • but not using the CSS or images it's supposed to
    • I view-source and click one of the asset links - get 403/Forbidden
    • play with chmod in the /static/splash/assets/ directory, now all look identical to files inside /static/assets/img/ - like -rwxr-xr-x 1 root dev 264 2012-10-22 21:02 icon-twitter.png - but the latter I can get in browser, and the former I can't!
    • ah, the issue was that the latter /assets/ container didn't have chmod a+rx
    • now when I hit the splash page I get the pretty version!
  • now request page that's public (no login needed) but dynamic - http://www.simplest-thing.com/resources_public
    • get permission denied for relation resources error. Suspect the user webpy doesn't have the appropriate rights inside psql. But off to bed....
    • yes, went into psql command-line with user webpy - could get in (as I did yesterday), but when I tried to actually do a select I got the same error message.
    • went back in as use postgres, did GRANT ALL PRIVILEGES ON DATABASE mf to webpy, quit, went back in as webpy, hit same wall.
    • list permissions: {{{ SELECT datname as "Relation", datacl as "Access permissions" FROM pg_database WHERE datname = 'mf'; Relation | Access permissions
      ----------+--------------------------------------------------------- mf | {=Tc/postgres,postgres=CTc/postgres,webpy=CTc/postgres} }}}
      • hrm what does that mean? Hm doesn't look like ALL PRIVILEGES were granted.
      • ah, that command doesn't do what it sounds like.
      • so did GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO webpy;
      • now it works! (in command-line)
    • now hit that resources_public URL again - it works!

Try to log in to web app.

  • Whoops No module named cryptacular.bcrypt. So do easy_install cryptacular then kill/restart uwsgi.
  • Try again: permission denied for sequence action_log_id_seq
  • I wonder if it makes more sense to make webpy the owner of the whole database?
  • Nah, just did Grant usage, select on sequence action_log_id_seq to webpy; for each sequence. (Oct26)
  • also had to easy_install markdown
  • discovered I was mis-using db.insert() in terms of returning the created ID value: you have to pass the db-sequence name associated with the ID.
  • also discovered I couldn't have a SQL query in code which used a literal in double-quotes, as psql doesn't like that.

Edited:    |       |    Search Twitter for discussion