Short-term goals: Visible Backlinks and Twin Pages for this WikiLog.
- encourage use of Smashed Together Words (contra Free Link-s) for Automatic Linking in other WikiEngines
- 2016 update: WikiGraph With FreeLinks
Building with WebPy to run on my Linode.
Data in 3 SQL tables: spaces, pages, mentions
Scraped page lists by hand from a number of my Sister Sites
- My WikiLog has 15664 pages.
- Ignoring Free Link-s cases and other bad URL-s (like Smallest Federated Wiki) for now, as I'm the client.
For Visible Backlinks, started with 2003 file giving graph of my space (only 2804 pages at that time!) - good enough for starting to build.
Jan20'2013 - get static pieces working together
urls = ( '/twinpages.js','twinpages', '/backlinks.js','backlinks',
class backlinks: def GET(self): web.debug('calling backlinks') resp = '' backlinks = db.select('mentions', where="space_name = 'Web Seitz Wiki' AND page_mentioned = 'Bill Seitz'") web.debug('backlinks: %s' % (backlinks)) if bool(backlinks): resp = 'BackLinks: ' for backlink in backlinks: resp += '<a href="%s%s">%s</a> | ' % ('http://webseitz.fluxent.com/wiki/', backlink.page_name, backlink.page_name) resp = resp[:-3] else: resp = 'No backlinks!' web.debug(len(resp)) return """ document.write('\ <div>\ <p>%s</p>\ </div>'); """ % (resp)
Feb22: clarify MVP plan
- focus on serving my WikiLog (and maybe Private Wiki)
- so no need to pass general params
- then move on to WikiGraphBrowser, since that shows the real power of WikiWord-s vs Free Link-s.
Feb27: get dynamic version of code working, using Http Referer and
Mar01: done styling content with inline div styles (how tacky).
Next: figure out how to parse all my current WikiLog pages for Visible Backlinks database.
- Mar10: done writing/testing locally
- Mar11: running against production.
- a few weird (MoinMoin) cases where
currentfilename contents point at a
revisions/file that doesn't exist! Always a tuggle in these cases as to how hard to work at solving the general problem vs just hacking the exceptions and re-running.
- have forward-refs for 15801 pages.
- a few weird (MoinMoin) cases where
Mar12: suck 96634 mentions into local db
- have some weird UTF8 issues, cleaned up file by hand
- convert SQLite to PostgreSQL format - see 2012-10-16-SettingUpHostingForSimplestThing
- get whole service deployed at Linode
Setting up hosting
/srv/www/simplest-thing.com/* - do chown/chmod bits.
/srv/.../app/start_uwsgi_app.shto parallel directory. Edit (port 9002).
/etc/default/uwsgi_wg, edit the latter to set
/opt/nginx/conf/nginx.conf - add a header section for hostless-domain redirect and a header for the www.wikigraph.net host that points to the right place and uses port 9002 for uwsgi.
code.py to server.
Restart nginx. Kill/restart the uwsgi app for Simplest Thing. Start the uwsgi_wg app for WikiGraph.
Try to hit http://www.wikigraph.net/test - get "uWSGI Error - Python application not found"
- hit same point in setting up Simplest Thing, but had already handled tweaks from that learning here, so don't know what issue is...
start_uwsgi_app.sh to call
uwsgi_wg instead of just
uwsgi. Kill/restart uwsgi.
Try to hit http://www.wikigraph.net/test - get 502 Bad Gateway. Realize didn't get the new uwsgi processes. Change the bash back. Get the "uWSGI Error" again.
Check the logs
- access.log - looks like status 500
- uwsgi_app.log - ah, notes that
import formsfailed (because not using that in this app yet, so hadn't copied it to server).
Tweak code.py, copy up, kill/restart uwsgi, hit page again. It does a redirect to
/static/... based on code copied from Simplest Thing, but I don't have that directory, so get 404 error.
Comment-out that check_authorized code, copy/restart/etc - now works!
Mar14: convert SQLite dump file to PostgreSQL
- copy converted dump file to server
- create db
- import dump file; do some
COUNTto confirm data in there
ProgrammingError: permission denied for relation spaces- just like in 2012-10-16-SettingUpHostingForSimplestThing
- do the
GRANTthing - now it works!
- copy MoinMoin template file to Blue Host (after various file games).
- It's working! Now displaying Visible Backlinks and Twin Pages!
Apr15: tweak to let my local Private Wiki call it. Automatically works for creating-new page process so I can see if Twin Page already exists in my WikiLog and decide whether to add a Private page or stay public.
- have WikiLog "page does not exist yet" page include Twin Pages.
- make way for backlinks-search to call db. Also change fulltext-search to call Google or maybe call WikiGraph.
- check scraper code for WikiWords with adjacent punctuation (like for italics)
- update lists
- support Free Link spaces (Phil Jones/SFW, Chris Dent/Tiddly Space...)
- 2016: WikiGraph With FreeLinks
Jul'2013 update: tweaked scraper code to catch some cases being missed before, re-ran
- 15963 pages
- 100451 mentions (6.3 mentions/page) (was 6.1 before - of course since both pieces changed it's hard to be sure...)
Twin Pages: pondering addition of Wikipedia
- 5M English articles, so don't feel like doing that
- 1000 "vital" articles, but suspect most wiki sites wouldn't have matches against very many of those (but maybe that's ok)
- could also do a hack, and always include a Wikipedia link that just generates a search...
- since Wikipedia is so unlike every other wiki, it actually sounds like a bad fit
Edited: | Tweet this! | Search Twitter for discussion