(2020-10-12) Building user management in WikiFlux

Building user management in WikiFlux so it will be multi-user.

using oldish stack from (2020-10-02) Setting Up Python On New MacBookPro

What kinda flow do I want?

self-service to create
- create "user"
- then pay, which then activates user which allows actual creation/editing (and links/redirects into the space)
login: hit link on any page, submit, get redirected back to self? Or root, I'm lazy.
view-vs-edit, privacy, etc.
- can view public page anonymously, have login link in menu as today.
- when viewing private page, or rendering Edit link for a view-public-page, have to check that the user is the owner of that space!

Oct12

pip install Flask-User → success
going to have to manually alter the users table schema.
- am I going to use the role feature? Or do something nastier to limit admin rights to my 1 user?
- probably keep it simple to start
- for users table, make a SQL script I can run later in prod. Actually 2 bits, because have to modify the existing record before adding constraints. They're saved in file, but I ran them in command-line.
edit models.py
launch app →

class User(Base, UserMixin):
 NameError: name 'UserMixin' is not defined

made some tweaks, changed to

    'No Flask-SQLAlchemy, Flask-MongoEngine or Flask-Flywheel installed and no Pynamo Model in use.'\
flask_user.ConfigError: No Flask-SQLAlchemy, Flask-MongoEngine or Flask-Flywheel installed and no Pynamo Model in use. You must install one of these Flask extensions.

so it's not seeing the db connection. I wonder if I need to change my other code to use db = SQLAlchemy(app) but that probably requires a bunch of other changes. Maybe need to find some more example Flask apps, conform my code?
Oct18 hacked the flask-user db_manager code to assume it's sqlalchemy

Oct18

added email-config values - use bill@simplest-thing.com
- run across suggestion to change settings at https://myaccount.google.com/u/2/lesssecureapps
it didn't like my secret_key, I had to muck around to satisfy it
now app is running!
paste in code for /members route
hit /members get sign-in page; enter info I had pushed into db before → get error, derp I added column email_confirmed should be email_confirmed_at
ALTER TABLE to fix column; hit again → UnknownHashError: hash could not be identified hmm not lots of help there
try registering new account bill@simplest-thing.com → AttributeError: 'scoped_session' object has no attribute 'session'
hack couple lines in flask-user to turn db.session. into db.
submit new-user again → fails because flask-user isn't creating id - null value in column "id" of relation "users" violates not-null constraint
while my code refers to field as auto-increment, nothing in the schema creates that.
- hmm should I use field type serial? How convert to that? Nope, user more SQL-compliant identity/generated
- once I settle on type, I can probably drop the existing id field then add the new type
- decide on this

Oct19

fix schema ^^
submit again - realize form is missing lots of fields I made required. Do I really want the user to have to enter them all to create an account? Probably....
derp they have a built-in template, I'll try the standard syntax to use that template!

Oct20

the built-in template only uses the couple fields, too. Gonna make local copy and add my fields. No still seems to be using their form.
change my route to /register, now get an error about form object for hidden_tag - research says that's for WTForms which is for CSRF protection, so I should be doing that anyway.
step back a sec - confirm app is working fine still with old hard-coded admin login, so I have a safe base to work form. So going to put in WTF/WTForms first. Get that working with old-model, then move forward.
pip install flask-wtf → I think it was automatically installed by flask-user
put in some code, but template is still unchanged.... it works
add in form.validate_on_submit() part - dies hard

Oct21

changed it to just use validate yeah it fails in there
ah probably forms class has to have fields matching what's in my manual form derp
still failing
derp had to put in {{ form.csrf_token }} in my form, boom!
(also confirmed making-new-page works)
next - back to Flask-User

Oct22

got to next error, I need to define the form class
- I can [extend}(https://flask-user.readthedocs.io/en/latest/customizing_forms.html#customizingformclasses) the RegisterForm class (to add fields), or copy/paste/fork. I'm going to try extending.
tried forking, couldn't figure out how to import validators to call, etc. so went back to extending
finally got form to load
but have no method to submit yet

Oct23

before I add any user records, I need to get that serial/generated thing working.
ok it's done (did I just do it, or the other day?) - did pgdump of schema
add more fields to schema, model, form, field
next: POST logic
unclear from examples how much is manual to get the form data, and then save to db
try to see if validate_and_submit() does the job, appear not.
ah, key method is populate_obj()
but fail: was a class (models.User) supplied where an instance was required? - just needed parens
now running but not inserting

Oct25

ah print user says the populate_obj() call isn't actually doing anything....
oh derp actual problem was that I wasn't calling commit!
now have issue with username is empty, despite form... (which then triggers uniqueness issue)

Oct26

try ripping out bits of form logic, still no change. Other fields (both built-in and added-by-me) are handled fine.

Oct27

realize I wasn't following the right approach to customizing form templates. Move my custom form, change code to map to it. Form loads, but now submit passes all null data.
derp some of the doc pages I've been using are for 0.6 instead of 1.0!
- different code change
- nope all still null!
uncommented form.populate_obj(user) - now getting fields except for username again
trying adding username to MyRegisterForm() - didn't help, took it back out
asking for help in github

Nov01

wondering if I should step back and align my code's structure with a more "standard" Flask model. But what the heck is that?
Do I have to adopt Blueprints?

Plan: I'm going to suck it up and upgrade to current Flask, then restructure in line with this

Nov03

pip install --upgrade Flask no complaints. Runs fine!
tutorial puts a venv inside the app's directory, then an app in there. I'm not putting the venv down at that level. But I will make an app subdirectory of my app's directory
make wikiweb.py above app with just from app import app
make __init__.py inside app with the app = Flask(__name__) and similar bits
rename old core file wikiweb.py to routes.py. Rip out the main caller bits, swap in from app import app for past app-import bits.
set environment variable from command-line
flask run → flask_user.ConfigError: Config setting USER_EMAIL_SENDER_EMAIL is missing. even though it's there in the routes.py file. Should probably move some config stuff around.
so jump ahead to section 3 - make config.py above /app/
flask run → launches! Hit page - works!
go back to section1, do pip install python-dotenv and make .flaskenv file. Still runs
section2: I already have a templates subdirectory with templates, and they are working since my pages are loading.
section3: I have WTForms, but maybe now's a good time to play with that directly for a non-user/login form...
create new page, leave body empty, submit → "Internal Server Error", see my form.validate() was false, with form.errors = {'body': [u'This field is required.']}

Nov04

ah, I see, they use validate_on_submit() so that if validation fails they can treat it like a GET and deliver the form. Would need nice error handling, but that kinda makes sense....
tweak to use that → happy paths work; missing-field silent (to user) but doesn't explode
I already had flash-messages in my layout template, now call flash if validate fail

Nov05

section iv - databases
switch to db instead of db_session, stop importing the database.py file that calls scoped_session, move config bits into config.py
launch → get error on user_manager = CustomUserManager(app, db, User) because db is not defined - suspect this is part of the kludging I did to get flask-user working, so will go back to their raw instructions....
follow their starter-app organization - move UserManager bits into __init__.py
now launches. Hit page → db not defined - still need a from app import db in routes.py
then need to comment out the db shutdown/remove bits
now working - to view page
edit → 'SQLAlchemy' object has no attribute 'commit'
change all the db.commit() to db.session.commit() → no error, but it also doesn't actually save the change!
added SQLALCHEMY_COMMIT_ON_TEARDOWN = True to config.py → edits seem to save, but
- when I re-edit the changed page, the body field has <p> tags?!?
- and saving a new page doesn't work - db.add(node) → 'SQLAlchemy' object has no attribute 'add'
make it db.session.add(node) → saves

Nov06: what's with the <p> tags?

seeing it for a bunch of pages
hrm it's actually in the db!
but only in the top few - weird have I had that bug for awhile?
no actually only a couple pages - viewing the page is enough to make it happen!
I think it's the "magic" library logic, combined with me over-writing the object value (but not explicitly saving that back!)
made that a new variable, called it in template → now all good
but now realize that all my edited pages have modified=null
it's failing my 2 action value cases, falling into a last case I just use for remote-POST, and therefore not setting modified.
printing full response.form shows I have 2 action entries, one of which is empty?!?
can't find where this is coming from
maybe having a form param named action is a bad idea? (It hasn't been an issue in the past. But maybe doesn't play well somewhere in magic now....)
ah, it's not just that field! It's happening with another field which is defined as HiddenField in my forms.py class?!? (Since my template is old, all the fields in it are explicit

Nov11

just make those field StringField instead of HiddenField - now setting Modified

Nov12

back to Flask-User (diverted as of Nov01)
null username issue again/still
pondering: why do I even need a username if I'm going to have people user their email for authentication? Maybe I should just make the field optional in the db, and move on?
part of my confusion is USER_ENABLE_EMAIL and USER_ENABLE_USERNAME config settings... do you have to set just 1 or the other? Does me having u_e_u set to False cause the code to ignore the form field?
set them both to True → it worked! Record created with value filled in!
but what will happen if I try to log in?
try, but was using random password and didn't save it, so rejected! Or is it something else? Actually page crash with UnknownHashError: hash could not be identified
try Forgot-Password feature → SMTP Connection error: Check your MAIL_SERVER and MAIL_PORT settings
paste/fork Flask-Mail SMTP server settings → email sends, I receive it
click link in password-reset email → 'SQLAlchemy' object has no attribute 'commit'

Nov16:

change flask-user sql_db_adaptor.py to use db.session.commit() - had changed that when I was first trying to get flask-user working weeks ago.
password-reset → works, receive the email. Click, get form. Enter new password (and save it this time). Saves successfully! (Though it also says Your account has not been enabled.)
submit email/password - says Your account has not been enabled. I'm guessing that's because db users.active is null
log in to the other account, which I had manually set to active=t, and login is fine.
noting that the tutorial series I've been following users Flask-Login instead of Flask-User grrr
will play with mapping of user to space before getting around to Stripe integration
oh heck just realized my whole user-namespace thing is based on subdomains, so I need to map a domain to localhost, can't use 0.0.0.0. Done.
found subdomain-route code
found filter AND syntax
realize that I never created any spaces upon creating the user.
Now wondering if I really need/want to have that.
if I force everyone to use /private and /wiki for their space names, it becomes unnecessary. (Or I could invent new defaults for everyone else, like /garden instead of /wiki, and handle mine as a special exception.)
hrm and spaces.path is redundant to users.subdomain
ah but spaces.id is foreign key in nodes.space_id, so I'd need to refactor that. And maybe I'll want to offer some people a 3rd space, etc.
so....
- in registration form I should ask for path and title for each of 2 spaces, and create them when creating the user
- and change my spaces.path to refer to the url-path rather than the subdomain
- have to re-order my logic to look up path->space, and then get its spaces.privacy_type

Nov17

had to put specific hostnames into my hosts file because it doesn't support wildcards for subdomains
had to put SERVER_NAME for domain into config (I think)
now I clearly have to go re-learn Flask-SqlAlchemy query/filter/join syntax
ah my tutorial series has a post that covers that
got the queries working (for view-1-page), now have to fix templates
have template content rendering, but wrapper bits are a mess - I think my static route handling fell apart in the refactor, though I thought I would have seen that affect sooner....
seems like static-path URLs aren't getting handled - connection refused - http://flux.garden:5000/static/bill2009_152.png - or, if I add a subdomain, then it gets picked up by one of my routes!
oh derp - when I added the subdomain cases to my hosts file, I left out the raw-domain case! So once I added that line to my hosts file, all good!

Nov18

hit a private page → NameError: global name 'is_anonymous' is not defined. Derp, that's User.is_anonymous
now recognizing have to login, redirects to /user/sign-in but that is still within the subdomain, so it gets grabbed by the route and fails - need to redirect to the raw-domain. Ah, should use url_for.... ah, arg for that needs to be in quotes. Now good!
tk - redirect or at least link to source page after successful login
reload private page - get redirected to sign-in then instantly redirected to sign-in-successful page
ah, it's not User.is_anonymous, it's current_user.is_anonymous - I thought that smelled wrong. Now it's good.
now get edit link to render. Change to use is_authenticated, generates link but it's wrong. Running into issues with using subdomain model.
include subdomain in call to render_template() → link in template url_for('node_edit', page_url=page_url, path=path, subdomain=subdomain) rendered to http://flux.garden:5000/private/2019-09-05-Jangly/edit?subdomain=webseitz
posted to stackoverflow.

Nov20

go no answer to url_for issue, so just making my own relative URL reference. Works.
hit edit link → getting matched to the page-view route instead of page-edit. Because haven't added the subdomain handler to the edit route+function. Also copy in other bits of refactoring from the page-view function.
hit edit → page container, but no form
rip out logged_in() conditional, fix functions to for new args, etc. → get form
submit edit → Saved! But redirect failed, added subdomain again
(and various other tweak) → edit saves, redirects to page-view all good
did the new-page variation → worked!
on to FrontPage: couple tweak → worked!
(actually, not true, it's not checking user yet, it's still the old logic there)

Nov22

hmm, I'm going to manually create the 2 space records for user2, make sure the page view/edit bits are working, then hit the FrontPage. (see Nov16 notes)
created spaces records - going to have issues with setting ID-increment-start-point for this table, too

Nov23

doing page-view - seems like favicon.ico is getting picked up by routes other than static, even though other static images aren't having that issue, and this file wasn't an issue with user_id=1.
Feels like I was failing-lucky before. Need to add url rule - but it's unclear where that goes (I had this in my old structure).
tried public/outer page-view (realized I was hitting a private, which adds complexity) - discovered Space lookup needed a lower() wrapper, fixed that.
now getting favicon error, but a different one, triggered by cascading redirects

127.0.0.1 - - [23/Nov/2020 11:12:29] "GET /favicon.ico HTTP/1.1" 308 -
127.0.0.1 - - [23/Nov/2020 11:12:29] "GET /favicon.ico/ HTTP/1.1" 308 -
[2020-11-23 11:12:29,086] ERROR in app: Exception on /favicon.ico/FrontPage [GET]

yikes cascading stupid, realize had space.privacy_type set to the path, not the privacy_type
then realized that the new user was id=16 not id=2 so the spaces weren't assigned right
now getting redirected to sign-in page
more derp I had set the wrong privacy_type for the space. Now rendering the page.
now realize I'm checking the user is logged in, but not that they own that space!

Nov24

check that user owns page to view-private-page
have editing working in appropriate cases also
trying FrontPage variations
my outer FrontPage loads fine (after tweaks)
new-space outer FrontPage: nope it expects sidebar pages to exist already
created fallback empty-strings → works! (and sidebar-items link to page-view/edit url to create)
new-space inner FrontPage also works
hmm have some things hitting a domain-top-root - What will be there? A pitch (SplashPage)? A list of (public) spaces? Both? OK, had commented out a route for that, put it back, tweaked that page to remind myself of what goes there, and how it fits into various flows....
next: RSS feed... working!

Nov28

title-search tweak → working

Nov29

space-owner-id=1 stuff has to be moved from global into something else....
focus on view-node route first → working (though style doesn't look right for other users, have to dig in after doing other routes)
did other page → good

Dec01

next - styling that looks wrong - different font from the body downward, though the H1 looks the same.
did view-source of each, pasted into HTML files did diff on them, found only the expected differences.
checked Safari as well as Chrome and got same result. Grrr
Inspect element - the "wrong" style is getting e-content style from default.css, while the normal/expected is getting that overridden by cherwell.css
oh derp, missed that line was buried in a chunk of "expected differences", for some reason that line was adjacent to a chunk of IndieWeb stuff I'm commenting out for non-me users. Moved → now good!

What's left?

Stripe → activation flow
- including creation of spaces - need to add fields to form and store them. Do that in the initial register form, or separate/later? Probably collect the data at register form, use them in creating the spaces, so don't need to store separately, and then just activation happens after Stripe.
- hrm suspect I'm checking various things about space/user, but not is_active
auto-post to WikiGraph service/db for Visible Backlinks. (Which belongs on a page of its own.)

Dec01

create stripe.com account
plan to use Checkout with subscriptions - use their hosted forms

Edited: 2020-12-03 14:52:05.472807 | Tweet this!

Bill Seitz