Webpy For Simplest Thing

Trying WebPy For Simplest Thing, coming from Pyramid For Simplest Thing.

May06'2012: Install into same VirtualEnv. Also easy_install pysqlite.

• take copy of mf.db created by Django.
• run code.py Hello World fine (May07)
  • also fine once tweak for template

try db section, using SQLite

• get unintelligible error
• realize I'm using wrong table name (users instead of Django-created auth_user), switch to using table tasks
• same error: <type 'exceptions.KeyError'> at /tasks
• hit SQLite command-line, sniff around. Doh! That table is named family_task. Tweak just the mapping of table-name, leave everything else as plain tasks.
• still get same error - looking more closely it's KeyError: u'/tasks'
• doh! Problem was in my urls list! Fixed that, now works!
• also map /users to auth_user table - listing also works
• next: figure out how to list query-results (from family_task where parent_id is null)
  • done, just use db.select with where argument

Next: combine some lists into an integrated page for my family-dashboard...

• getting single-record - use db.select()[0]
• do simple db.select to include multiple related lists (May08)
• next: isolate base layout template bits
  • first line of specific template needs to be the $def with, then 2nd line can be $var title: (May13)

Next: work in SQLite to figure out proper join to generate key list for that page

• this is going to be a self-outer-join!
• overall master tasks: {{{ SELECT id, name, description, sort_order FROM family_task WHERE master_id IS NULL AND parent_id IS NULL }}}
• status from family-specific list: {{{ SELECT id, master_id, status FROM family_task WHERE parent_id IS NULL AND family_id = $family_id }}}
• These 2 lists will intersect, but there can be exceptions on both sides. So will want a FULL OUTER JOIN
• SQLite does not support a FULL OUTER JOIN!
• So it looks like I need to do a UNION ALL or UNION.
• Actually, there are enough weird cases I'm thinking it makes sense to just do a couple separate queries and then use code to combine the results.
• hmm, in building up custom result-set list, discover IterBetter instance has no attribute 'append'. So immediately convert db.select() result into a list.
  • then when rendering an item in layout, have to use obj["col"] instead of obj.col
  • should I improve consistency by building a list of objects instead of a list of dictionaries? Nah.

Next: bring in BootStrap CSS, then figure out how to generate Drop Down of status options for each goal and have changes do AJAX update.

• make a static folder, copy BootStrap assets folder inside it
• copy top and bottom bits of BootStrap starter-template.html into layout.html, tweak some paths to /assets
• looks a little nicer. Going to delay Drop Down plus more cosmetic stuff until after getting authentication stuff in.

Next: user authentication (Web Authentication), and maybe sessions.

• yikes, this hasn't been written yet.
• seems like a number of people are using jpscalleti's code so I'll give that a try. Of couse I still have auth-related tables in db from starting with Django For Simplest Thing. So I guess it's time to drop those tables and bring in the new ones...
  • dump mf.db to mf.sql, edit sql file to remove Django bits. Also remove family_ prefix from every table (and tweak code that references those tables).
  • jpscaletti's schema generates some errors, so have to tweak
    • AUTOINCREMENT instead of AUTO_INCREMENT
      • add AUTOINCREMENT to other id fields
    • remove comments
    • restore using sqlite3 mf.db < mf.sql
  • stick his code lines up where they belong. Launch. (May14)
  • error name 'app' is not defined because of mysession = web.session.Session(app, web.session.DiskStore('sessions')) - need the rest of the sessions stuff. Copy from here, now existing pages load fine.
  • trying hitting /login URL - get 404 (not found)
  • try adding param to settings dict, no change. Set settings back to empty dict
  • try adding @auth.protected() to a page method, hit that URL. It redirects me to /login returning 404 again.
  • try copying login.html template to my app's templates directory - no difference.
  • I'm not the only one who's had this issue.
  • emailed jpscaletti to ask for help. Try mucking around with pdb but can't figure out what to do, so just going to wait...

Back to some BootStrap CSS work...

• Copy header ribbon from fluid example.
• Copy sidebar-nav section from fluid
• Use <table class="table table-striped">, add thead/tbody bits
• next: status Drop Down-s - ideally want to do AJAX db update when user changes the drop-down status value for any given Goal. (May15)
  • Looking through Head First Jquery, p427 (p465 of the PDF).
  • but going to revisit user-authentication first...

User authentication - continuing from above. (May15)

• Haven't heard from jpscaletti.
• David Montgomery, who had similar problems last year, never got them solved.
• I'm going to try a couple hacks and see if they work...
• though maybe I could a map for DBAuth.login in urls but DBAuth gets called after web.application(urls...) - maybe someone smarter can make that work...
• so try more-manual approach
  • add login map to urls to my own login class, have that render my local copy of the login.html template. Result: yes, now get a nice login form. But submitting it does nothing.
  • smells like a slippery slope of copying and pasting huge amounts of stuff.
• I guess I'll just have to start from scratch (and then steal tiny-bits-at-a-time from that code).
  • I can pass an object or a dictionary to a template, but it seems like I can't use that in the base-layout template.
    • When I tried passing a dictionary, then referencing $content.user_block["user_name"] in layout.html, I got error <type 'exceptions.TypeError'> at / - string indices must be integers and found local vars content = <TemplateResult: {'__body__': u'\n<h1>Welcome</h1>\n<p>Sorry</p>\n', 'user_block': u'{'user_name': 'Sorry'}', 'title': u'Personal Finance for Busy Parents'}>
    • When I tried turning this into an object instead and referenced $content.user.user_name I got error <type 'exceptions.AttributeError'> at / - 'unicode' object has no attribute 'user_name' with local vars <TemplateResult: {'__body__': u'\n<h1>Welcome</h1>\n<p>Sorry</p>\n', 'user': u'<code.User instance at 0x101e2e290>', 'title': u'Personal Finance for Busy Parents'}>
    • but wiki example shows that base layout can have logic, so I don't have to be totally stupid about it.
      • yes now have some toggling logic in the base layout.html (May16)
  • so, to have the login/register logic bits in layout.html, it looks like (May17)
    • every view-generating method needs to
      • be defined like def GET(self, user_name = None): (to set default user_name)
      • include {{{ if session.has_key("user_name"): user_name = session.user_name }}}
    • the template must start with something like $def with (form, user_name) so it can pass that user_name to the base layout.html
    • the base layout.html then uses content.user_name
  • or maybe I can simplify by copying this approach
    • before doing all that decide to revive my use of Mercurial. Do init/add/commit of this directory. (May19)
    • Move if session.has_key("user_name") to top level, move render = web.template.render('templates/', base='layout', globals={'user_name': user_name}) right after that, take any def(user_name) out of templates, change base template to use $user_name instead of $content.user_name. Looks like it works! (May23)
• Playing with register form.
  • Now that I'm controlling this more directly, what should the user think of as his ID? Probably email address. (If I wanted to be fancy, I'd let a user associate multiple email/password pairs with his account, since users tend to forget what address they used...)
  • realize that dbauth library didn't have the form/wrapper to call the createUser() function, so start creating HttpPost target.
    • add CSRF token bits
  • read some Jeff Atwood security notes (see OWASP page), decide to use BCrypt method from dbauth
  • made bunch of changes. Now when hit any page get <type 'exceptions.TypeError'> at / - 'dict' object is not callable triggered by app = web.application(urls, globals()) - is that an issue related to setting a globals dictionary to be used by base layout?
    • hrm, like this guy, when I restarted the server the problem went away! (May24)
      • except that every time I have some other little error, when I fix and then reload a page, I get that same error message and have to restart the server. Maybe something with session?
  • progress - current status
    • need to recheck whether I'm handling sessions right for reloading/debug mode.
      • yes, I'm doing that fine. Though I was concerned about possible confusion between web.config used by sessions code vs config used by dbauth stuff I copied, to I changed the latter to config_auth. Still get that dict-not-callable error after fixing some other error.
    • I know I'm not actually checking for the CSRF token.
      • Urgh, first had to fix example to have form include straight param by name, not as function call. <input type="hidden" name="csrf_token" value="$csrf_token"/>
      • submits always getting rejected because session.pop('csrf_token',None) always gives None.
      • But if I take it out and just look for session.csrf_token then when there isn't one I get a <type 'exceptions.AttributeError'> at /register'Threaded Dict' object has no attribute 'csrf_token' - so I definitely need a safe wrapper for that get, so I put back in the session.pop bit (behaving same as when it was there before)
      • decide to take the debug/reload bit out of the equation, so put in a web.config.debug = False - still getting same error!
    • if try to insert dupe, get error msg - need to pass it to user
      • this raises general issue of returning user errors - people are recommending to use AJAX form handling.
    • even though (supposedly) adding user_name to session, it's not appearing anywhere. Is there a way to display the session data?

Feeling like solving too many generic problems here. Raise the idea of taking an existing sample app and adding lots of these cookbook pieces, hosted on GitHub for moving forward and sharing. (May28) So go on tangent: Extending Webpy Blog App With Cookbook Features.

• May30: hit wall with CSRF just like for my own app. Ask people for help over there.

So on this side, turn off the CSRF protection for the form. (Also have the session-with-reload stuff commented out, using debug=False instead.) Confirm can now do a register.

• May31 update: CSRF resolved - put back in.

But user_name still never showing up anywhere. Let's try something simpler: modifying the /hello/{user_name} handler to set session.user_name.

• playing around gets me toward thinking I'm assuming that the main body of code gets executed for every hit, but that's probably a big mistake....
  • change from {{{ globals = {'user_name': user_name, 'csrf_token': csrf_token()} render = web.template.render('templates/', base='layout', globals=globals) }}}
  • to {{{ render = web.template.render('templates/', base='layout', globals={'user_name': user_name, 'csrf_token': csrf_token()}) }}}
  • this makes no difference
  • (update: changed call again based on CSRF changes above - still no change in outcome)
• even simpler case: have a /test handler which outputs via return without a template. Have that include session.user_name and it properly outputs the value I set with /hello/Bob

Jun13: integrating learnings from WebpyForSimplestThing re sessions, CSRF, etc.

Jun23: moving register/login logic forward

• hmm, user_password doesn't look like it has the salt stored in it
• Jul22: hmm even weirder it doesn't seem like the expected BCrypt function is even being called! (Something weird in the indirection being used from dbauth.) So call that hashBcrypt() function directly, and get an error at import bcrypt! So have to figure out where that is.
  • find bcrypt folder inside cryptacular directory, but it's possible I downloaded that last week - really not sure. But it's not getting found either way.
  • ah, looks like I got cryptacular with/because-of the shootout demo app for WebPy.
  • clearly the dbauth code wasn't using cryptacular. So changing the stuff I stole to use cryptacular.
• register is now storing correct-looking hash value in db for user_password
• and login is now working right. Also displaying error messages. But those messages aren't red, and it's not doing a back to the form which would save my values. So no ideal, but ok for now.
• next: back to actual app logic. (Getting this auth stuff put back into Extending Webpy Blog App With Cookbook Features will have to wait.)
  • also need to work on some authorization code!

Jul24: app logic

• db.select() returns an IterBetter object, which has no len()
  • if you just want to check where there are any rows returned or not, test bool(rows)
  • but if you want to check for some number of rows other than 0 or 1, then you need to convert with rows = list(rows) then num = len(rows).

Jul30 - rendering some textarea/blog user-data with Mark Down, following notes here, though it looks like safemarkdown is in web.utils. Hitting a wall, [posted](https://groups.google.com/forum/?fromgroups#!topic/webpy/Qeg Ng Jy Iv7c) a question to the forum.

• update: got answer that works. You should use $:safemarkdown(note.description) - $foo always escapes the value. $:foo renders it verbatim.

Aug01: form.fill(record) fills in current record values for an Edit form.

Aug02: by default, WebPy forms do an HttpPost back to the same page/form URL, because they have no action specified. This becomes an issue when you want to have more than 1 form on a page, since they both end up posting to the same URL. But actually, this is just a convention, because the Form code doesn't generate the form container tag at all! So by convention people don't put any action in the tag, so it falls back to the page URL. So you can easily manually add an action pointing elsewhere for 1 of those forms.

(Have been working on a lot of app-domain-specific stuff, so nothing bloggable.)

Sept25: using an "ApplicationProcessor" to check for all my non-static URLs that user is logged in.

Sept28: generating/sending HtmlEmail

• make a render_email() variety that doesn't use the background template wrapper
• remember that any links need an absolute reference
• very simple to use GMail to send the emails.

Oct12 - make forms look nice by modifying WebPy's /web/form.py to use BootStrap styles.