WebSeitz/wiki

Flask For Wiki Engine

Want to learn Flask, so using it to build little WikiEngine, which might end up hosting my WikiLog. I'll call the engine WikiFlux. (Update: now available as hosted FluxGarden.)

2014 - start

Nov03'2014 - start, using Flask

Nov04

  • find Explore Flask ebook, note it recommends first reading official Quick Start and Tutorial
  • pip install flask-debugtoolbar
  • going to plan on using blueprints for organizing
    • grr very confused how to organize things the "right way" up-front.
    • will probably take "divisional" approach, smells closer to MicroService separation
    • so wikiweb is the virtualenv, will contain just this one big app, so not adding an extra layer of directory. Treating first blueprint as space so have wikiweb/space/ which contains templates, static, etc.
  • rough Data Base schema
    • spaces: id (int primary_key), path, title, owner_id, privacy_type (wikilog, privatewiki, ...)
    • nodes (pages): wiki_name_lower (unique primary_key), space_id, wiki_name (unique), body, created, modified
      • clarifying - id/name issue - goal is to have certain level of Case Insensitivity for url/name
        • wiki_name_lower is unique, has associated definitive Smashed Together Words/Camel Case wiki_name
        • if ask for Camel Case url, it first looks for that exact match
          • if fails, looks for lower(url) against wiki_name_url, if finds that match then does redirect to wiki_name
    • users - want to support IndieAuth if that makes sense, but what if this is the user's primary site/domain?
    • hrm trying to run things break stuff all over the place, smells like really need a directory level for the app. So move all the app bits into wikiweb/wikiweb/
      • have wikiweb/run.py, wikiweb/wikiweb/__init__.py and wikiweb/wikiweb/instance/config.py
      • try python run.py and get
Traceback (most recent call last):
  File "run.py", line 2, in <module>
    from wikiweb import app, db
  File "/Users/billseitz/Documents/djcode/wikiweb/wikiweb/__init__.py", line 8, in <module>
    app.config.from_pyfile('config.py')
  File "/Users/billseitz/Documents/djcode/wikiweb/lib/python2.7/site-packages/flask/config.py", line 128, in from_pyfile
    with open(filename) as config_file:
I O Error: [Errno 2] Unable to load configuration file (No such file or directory): '/Users/billseitz/Documents/djcode/wikiweb/var/wikiweb-instance/config.py'

* use pdb to find that within `from_pyfile()`, `self.root_path = '.../wikiweb/var/wikiweb-instance'`

Nov06

  • set instance_relative_config=False, move the config file
  • starts to run, but then ImportError: No module named psycopg2. Try to do pip install psycopg2 but something fails there, too.

Nov12: let's start again with baby steps

  • hello.py still runs
  • let's step all the way through the Tutorial
    • (must have missed the part about username=admin, password=default)
    • now it works, using SQLite
  • now try to get it working with PostgreSQL using the Mac postgres.app and SqlAlchemy
    • hmm doesn't seem directly comparable - Flask tutorial code uses db_connect() which isn't the same.
  • so let's take a step back and get it working with SqlAlchemy talking to SQLite.
    • able to get init_db() working
    • now have app working - should put this on GitHub before trying to get it working with PostgreSQL

Nov13: get flaskr.py working with PostgreSQL

  • change 1 line: engine = create_engine('postgresql://localhost/flaskr', convert_unicode=True)
  • run -> ImportError: No module named psycopg2... so {{{ pip install psycopg2.py Downloading/unpacking psycopg2.py Could not find any downloads that satisfy the requirement psycopg2.py Cleaning up... No distributions at all found for psycopg2.py Storing debug log for failure in /Users/billseitz/.pip/pip.log }}}
  • ok not just me
  • oh wait didn't need that .py - just pip install psycopg2 {{{ Please add the directory containing pg_config to the PATH or specify the full executable path with the option: python setup.py build_ext --pg-config /path/to/pg_config build ... or with the pg_config option in 'setup.cfg'. }}}
  • note recommendation here to install while not having the VirtualEnv be active - use bin/pip...
  • find /[virtualenv]/build/psycopg2/setup.cfg
  • uncomment/edit line per this: pg_config=/Applications/Postgres.app/Contents/Versions/9.3/bin/pg_config
  • try bin/pip again, different fail {{{ Installation Error: Command /Users/billseitz/Documents/djcode/st/bin/python -c "import setuptools;file='/Users/billseitz/Documents/djcode/st/build/psycopg2/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --single-version-externally-managed --record /var/folders/g5/d_mhycbn075d7yptvn9skd440000gn/T/pip-ZSGUoq-record/install-record.txt --install-headers /Users/billseitz/Documents/djcode/st/bin/../include/site/python2.7 failed with error code 1 in /Users/billseitz/Documents/djcode/st/build/psycopg2 }}}
  • try this: PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.3/bin/ sudo easy_install psycopg2
    • at least 1 "warning" generated, but maybe ok?
  • re-activate VirtualEnv, launch flaskr.py - still fail
    • note Using /Library/Python/2.7/site-packages/psycopg2-2.5.4-py2.7-macosx-10.9-intel.egg - so install happened in the main python, not the VirtualEnv
  • realize (comment inserted above) that was working in the wrong VirtualEnv
  • go into the correct VirtualEnv's /bin/, try again PATH=$PATH:/Applications/Postgres.app/Contents/Versions/9.3/bin/ sudo easy_install psycopg2
    • get warnings
    • do python flaskr.py - no errors!
  • hit root page - get Operational Error: (Operational Error) FATAL: database "flaskr" does not exist
    • quit flaskr, launch python, try {{{

from database import init_db init_db() }}}

  • same error
  • go into PostgreSQL's command-line window, do create database flaskr;
  • do python... init_db() - no error!
  • python flaskr.py - hit root page - works!
  • enter records - works!
  • push tweak to GitHub

Nov14: back to starting on WikiWeb....

  • rip up most of the blueprints stuff, copy/tweak bits from flaskr.py
  • run CREATE DATABASE wikiweb
  • run init_db() - create tables
  • do some tweaking - have enough working that it gives you form like Flaskr to enter wiki_name and body, and saves it, and lists them all. No WikiWord catching, no Mark Down rendering, nothing.
  • save to Git. Not worth putting in GitHub yet.
  • so, to launch

Nov16: can view single-node Page (but just raw content still), with not-found/edit/create link if fail (but link is wrong). Commit (Nov17).

  • note that tried using node = Node.query.filter_by(wiki_name=wiki_name).first_or_404() but got AttributeError: 'Query' object has no attribute 'first_or_404' - so just used first() which seems fine anyway, since I check for returned object in the page template.

Nov17: Common Mark rendering: Looks like Roland Shoemaker's version is best choice for now. Though his hasn't been updated in a month, while the spec has had a number of changes since then (but I'm not sure how significant they are). Will give it a try. Update: working!

Nov18-20: Correct Edit-page link URL. Correct handling for view-page of non-existing page. Commit

Nov21: submitting page-change.

  • Currently getting "sent a request that this server could not understand".
  • fixed that, now have to distinguish creating from editing. Update: done. (Also now update node.modified on update.)
  • Get pagename in title tag! Update: done
  • Get spacename/link in title/header (just static for now, can make db look up in future version). Update: time to move config items to file. (Also want to include IndieAuth tag.)
    • actually didn't move to config file, just realized that config items needed all-caps names.

Nov22: Some Web Design work

  • want something as clean/lite as possible
  • almost like a Card (CardDeck) (but infinite length)
    • and not literally, since Card design for desktop is mostly about displaying multiple cards (though that's semi-relevant for the Front Page maybe)
  • very little at top beyond a ribbon of identity
  • maybe collapsed menu
  • because there's so little to mess with, probably don't want to bother with BootStrap.
  • hmm what does Google Glass HTML/CSS look like? No, that's definitely oriented toward small fixed chunks.
  • Bah, just went looking for super-clean blog site, very happy with Ben Werdmuller's, so stole it! It uses BootStrap, so back in that game. Re-arranged some items, ripped out some bits, pretty happy with how it's looking. But still using lots of Ben's link tags inside, plus not even sure it validates after having mucked with it....
  • Swapped in my photo, made favicon.ico, removed many tiny-icon image links.
  • Suppress certain meta tags in master layout when not in single-node page... done.
  • Bring any remaining Ben W assets/links local. Done
  • Search form in menu: make cheesy hack to redirect to Google.

Next: Automatic Linking

Got distracted by Auto-Complete, plus busy with life.

2015

Aug20'2015 Back to pondering Automatic Linking - 3 places/times it could be done

  • before calling Mark Down - generate Mark Down-format links around names
    • has the advantage of the rejection cases all being inside brackets, thus simpler to test
    • but what about checking whether a page exists? How pass that info along in the Mark Down link?
  • inside Mark Down/Common Mark library - nasty
  • after Mark Down conversion - seems like potential for weird issues, like tag ids/parameters that are Camel Case or AllCaps
  • so leaning toward first method!
    • also remember to handle other-wiki case like WikiWikiWeb:PageName
    • and to separate words in tag-text

Duh, realize I have something that works pretty well already in WikiGraph scraping code!

  • grr feel like there's some bug trapping/stripping a space when finding/replacing a WikiWord, so I'm just using a hack to add it back.
  • Made "mistake" and output HTML link instead of Mark Down link. Then, because it worked fine, realized/remembered that Mark Down will just pass through HTML bits fine. Which means I can do the check-if-page-exists part in this first pass, and generate the appropriate link type up front. So that's nice.

Aug25: last code edits (hrm last Git commit Aug20). Note had never dealt with issue where Smashed Together Words are inside a URL, or in linked text. And RegExp seems very nasty way to be catching those conditions.

Then revisit Automatic Linking, conclude that it's time to drop Smashed Together Words and embrace the Free Link. Last notes were Sept27.

So design plan is (Oct13):

Start coding changes (Oct13)

  • ack Git is broken! Some of recent stuff has whacked it. Moving Stencyl stuff around, or what? Well, will move along without it (because offline on train)
  • unlike plan when using Smashed Together Words, going to do WikiWord catching after running Mark Down.
  • get basic code working for simplest case
  • get tests.py set up to import/call/test functions from within wikiweb.py
  • next
    • rip out strange function at top of tests.py? Done
    • handle making URL for Free Link - currently just stripping spaces, have to (a) strip other punctuation; (b) capitalize words(?). Done
    • check for whether page actually exists (and tweak styles). Done
      • but need redirect when fails exact-match but hits case-insensitive-match - Done Oct16
    • handle BlogBit pages - how write WikiWord, how render a URL. Done
    • fix above for weird cases: Schlep Work vs Schlep-Work vs SchlepWork; also McString and ACRONYM: Done Oct17
    • check regular outside links, incl style. Done Oct17 but ugh [label](url) (Converting Moinmoin Text To Markdown]]
    • make bigger Text Area!!!!
    • handle InterWiki links WikiWikiWeb:WikiStandards - ugh requires db of spaces (update: hack)
    • then fix Git! Done Oct20
    • then WikiGraph widgets

Oct20 fix Git, do commit. 2015-10-18-LaptopDevProblems

Bigger Text Area: Oct21 - hmm not as wide as it should be. Can't find CSS driving it. Ah, it's in bootstrap.css and set to 206px wide. Change to 606px. But the height in rows is driven by node_edit.html template.

InterWiki: realize I can just do a hack for now: hash in code, not db.

  • code done Oct21
  • To Do: add/fix spaces URL hash; improve error handling?

Front Page: correct order; limit number; render bits; sidebars

  • already was showing just first line from every node
  • now render Common Mark and WikiWord-s. Done Oct21
  • order by date. Done Oct21
  • show "more" link. Done but needs fixing (only show if needed, and put inline). Done Oct22
  • next: show 20 at a time
    • use slice? no, use age-of-oldest-on-page to jump to next
    • need to handle starting point, plus case when run out of pages - Done Oct29
  • next: style Front Page
    • first tweak style to regular page - too bland since I don't have background image like Ben W. Color bar at top? Look at kottke, plus my Tumblr accounts.
    • also need to revisit storage of "pretty" page name, because need to start rendering that instead of just wiki_name! (I'm back to feeling like I should just put it in the nodes table, because going to need it so often with the node it seems silly to have to go query for it, esp for lists like Front Page)
    • never mind - this isn't important enough for now

Nov10: set space-delimited node.title

  • db structure
  • forms, h1, etc.

Nov11: sidebar bits on the Front Page

  • define list of nodes for sidebar as per-space list; render list: done
  • next: style Front Page: done (BootStrap rows/spans, plus panels)

Nov11: RSS feed

Nov11: JavaScript widgets: WikiGraph, Sister Sites, DisQus. Done.

Nov12'2015: starting Converting Moinmoin Text To Markdown (plus scraping old, plus posting new)

Plus couple misc things to tweak soon

End-2016

Nov26'2016: flip DNS to make this my new live WikiLog.

Dec'2016-Mar'2017: start scraping InstaPaper.

Mar14'2017: validate rel-me and h-card and h-entry and h-feed for IndieWeb.

Edited:    |       |    Search Twitter for discussion

Made with flux.garden