(2020-10-02) Setting Up Python On New MacBookPro

Setting Up Python On New MacBookPro - from (2020-09-23) Time To Replace MacBookPro

Goals

  • Python 2 and 3
  • OpenSSL current
  • clean up some of my project/directory structures
    • hierarchy based on WebAppFramework; peer directory for non-WebApp pieces - root at Documents/code/py2 (not /djcode/) - will do pyenv local 2.7.18
      • flask
        • wikiweb
        • wilson (coachbot webapp)
      • webpy
      • other(?) (or just pop these up a level? No, want single virtualenv shared across them)
        • wilson (coachbot command-line)
        • autopost
        • insta_repost (move up out of wikigraph)
        • TwitterBots
  • hmm do I want to plan on Anaconda, anticipating stepping up ML work in CoachBot? Nah let's keep it simple.

Mac OS-X 10.15.7 - Catalina

PyEnv vs VirtualEnv vs Ana/conda vs ???

Oct03'2020

  • going with pyenv, following these instrux
    • homebrew
    • pyenv
    • pyenv -> profile
    • python --version -> Python 2.7.16
    • xcode-select --install
    • brew install openssl readline sqlite3 xz zlib
    • pyenv install 2.7.18
    • pyenv install 3.8.5 (tried 3.8.6 but was just released a week ago, and pyenv doesn't have it)
  • hmm what Flask version am I running?
    • Linode: 0.11.1 (on python 2.7.6)
    • old MacBookPro: no Flask! I suspect my thrashing attempts to deal with OpenSSL a couple months ago wiped it out or something.... Onward...
    • so what should I install on this new laptop?
      • Jump to latest, get my code working, then update Linode to match?
      • Match Linode? Going to take this approach, seems less risky (if I can install the bits I want)
  • mkdir code/py2 (not quite), go into dir, pyenv local 2.7.18
  • Flask docs suggest virtualenv, so do that
    • need pip: download get-pip.py, do python get-pip.py
      • WARNING: The scripts pip, pip2 and pip2.7 are installed in '/Users/billseitz/Library/Python/2.7/bin' which is not on PATH.
    • python --versionPython 2.7.16 wtf?!? Why is it using the system Python?!? Ah, close, start new Terminal window, now it works (I had done a command from earlier instructions that should have used the updated .zprofile but maybe that didn't work
    • do python get-pip.py again
    • pip install virtualenv
    • mkdir flask, cd flask, virtualenv venv
    • source venv/bin/activate This is key thing to run every time I'm in this project.
    • pip install Flask==0.11.1 (per this) - seems like it worked
  • copy the project directories over under /flask/ from old laptop
  • python hello.pyRunning on http://127.0.0.1:5000/ - hit the URL, it works!
  • go into /wilson/ project directory, python wilson.py get errors about missing sqlalchemy (pretty sure I don't use db in that app, but probably have stuff installed anyway) (not going to worry about that for now)
  • switch to /wikiweb/ directory, python wikiweb.py get same sqlalchemy alert
    • any requirements.txt file to give any clues - nope
    • error also noted flask.ext.sqlalchemy is deprecated - so shall I install the newer flask_sqlalchemy and hope I can move forward from there? OK let's try that.
    • pip install -U Flask-SQLAlchemy → good Successfully installed Flask-SQLAlchemy-2.4.4 SQLAlchemy-1.3.19

Oct04 continuing

  • pip install psycopg2Error: pg_config executable not found
    • launch my Postgres.appempty data directory → press Initialize button, it creates folder
    • sudo find / -name "pg_config" -print/Applications/Postgres.app/Contents/Versions/13/bin/pg_config
    • add PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH" to bottom of .zprofile
    • pip install psycopg2 successful
  • python wikiweb.py → issue with from werkzeug.contrib.atom import AtomFeed, No module named contrib.atom
  • at some point werkzeug dropped atom
  • what version of werkzeug do I have?
  • find → /code/py2/flask/venv/lib/python2.7/site-packages/werkzeug, cat __init__.py__version__ = "1.0.1"
  • per prev link, pip install Werkzeug==0.16.1
  • python wikiweb.py → success!
  • hit the URL, get empty page, plus error about database does not exist. As expected.
  • back to python wilson.py which doesn't use db. ImportError: No module named CommonMark
    • pip install commonmarkSuccessfully installed commonmark-0.9.1 future-0.18.2
    • run wilson.py again, same error
    • hmm going to let this slide for now, it may come up in wikiweb since I use CommonMark there, too.
  • migrate data, use pg_dumpall
  • python wilson.py no empty-db error! but get the CommonMark error!
    • grr I see it in /site-packages/; I've started new terminal window, gone back into venv, etc.
    • change my code's references from CommonMark to commonmark
  • launch again - ImportError: No module named pyparsing
    • pip install pyparsing
    • FrontPage renders!
    • single-page renders!
  • do login - when submit id/password it's just sitting there... but if I go back and reload page I was on, I can see I'm now logged in. And can switch over to private pages. And can edit/save.
  • make CommonMark changes to wilson, can now run that fine.

Oct04: on to webpy - it's not dead! The latest 0.61 release supports Python >= 3.5. Version 0.51 is the last release with Python 2.7 support.

  • mkdir /code/webpy, cd webpy, virtualenv venv, bin/activate
  • old mac was using 0.37
  • pip install webpy==0.37ERROR: Could not find a version that satisfies the requirement webpy==0.37
    • hmm see it listed here
    • ah it's pip install web.py==0.37 → success
  • copy over old directories of webpy code - except wikigraph
  • cd mf, python code.py → no error
  • hit http://0.0.0.0:8080/, get splash page
  • hit some other pre-login pages, even those pulling from db work fine!
  • I don't remember my login, so I'm not even going to bother.
  • same for mss
  • hah realize that both of these apps uses MySql for their local versions
    • look up my admin login info
  • hit mf login - submit id/pw → No module named cryptacular.bcrypt
    • pip install cryptacular==1.4.1 (that's the version on old laptop)
    • relaunch server, hit page, get private db-driven page fine
  • launch mss, hit/submit login, get db-driven private page

Oct05: wikigraph -

  • copied over "core" files from old laptop
  • launch server: page comes up, but D3 graph in the middle doesn't show anything; decide to try the key widgets first
  • tweak local wikiweb to call local wikigraph, hit wikiweb page
  • error, in referer_parse function - I have some cases already to handle local URLs etc, but
    • it's coming through now as 0.0.0.0 instead of localhost
    • check both cases → now works!

Oct06: look at side-bits that were buried inside wikigraph

  • create /other/
  • organize wikigraph muck into clean sub-folders, copy over; also copy over the other non-server code directories
  • virtualenv venv, activate
  • going to try Instapaper-related bits before TwitterBots (but copied the twitterbot folder over)

Oct06 InstaPaper

  • (tweak directories used in code for pipeline files)
  • python insta_repost.pyImportError: No module named oauth2
  • old laptop's oauth2 is v 1.9.0.post1, which still has a reference page
    • then there's python-oauth2 1.1.1 whose page says If you are looking for a OAuth 2.0 library to integrate into your Python application, I recommend oauthlib.
    • go for backward compatibility: pip install oauth2
  • python insta_repost.py - success! Edit a highlight file....
  • python autopost.pyImportError: No module named requests
    • old laptop has v 2.18.4
    • pip install requests==2.18.4 → success
  • python autopost.py → success!

Oct06 TwitterBot

  • cd sycorax, python nudge.pyImportError: No module named twitter
    • old laptop has v 1.3.1
    • pip install twitter==1.3.1 → success
  • python nudge.pyAttributeError: 'module' object has no attribute 'Api' (from api = twitter.Api())
    • ah, it's python-twitter I need (reading from TwitterBot page); pip install python-twitter==1.3.1
  • python nudge.py → same error grrr
    • pip uninstall twitter
  • python nudge.pytwitter.TwitterError: [{u'message': u'Invalid or expired token.', u'code': 89}]
  • python get_access_token.pyImportError: No module named oauth.oauth see TwitterBot notes.
  • or should I try jumping to BotFriend?

Nope, I realize TwitterBot stuff isn't actually a priority, esp since it's already been broken for a year. So just going to let this alone and move forward on the rest of the laptop-migration tasks.

(Late Oct07 - did WikiGraph update which is further confirmation of some things working, though that's pretty decoupled from servers/etc so it doesn't say much...)

Nov14 - laptop rebooted the other day. Now insta_repost isn't working - giving me ImportError: No module named oauth2 despite prior setup Oct06. Do pip install oauth2 again, now can run code.

(2021-02-07) Migrating to Python3


Edited:    |       |    Search Twitter for discussion