Bcrypt Hash Returns TypeError("Unicode-objects must be encoded before hashing") and Invalid Salt

Question

I've looked at all the StackOverflow questions related to this but I just cannot seem to figure this out. When I hash a password, and check it against itself, it returns the TypeError "Unicode-objects must be encoded before hashing" with the current code:

from scripts import tabledef
from flask import session
from sqlalchemy.orm import sessionmaker
from contextlib import contextmanager
import bcrypt

(Unrelated Python code...)

def hash_password(password):
     return bcrypt.hashpw(password.encode('utf8'), bcrypt.gensalt())


def credentials_valid(username, password):
    with session_scope() as s:
        user = s.query(tabledef.User).filter(
            tabledef.User.username.in_([username])).first()
        if user:

            return bcrypt.checkpw(password.encode('utf8'), user.password)
        else:
            return False

When I try and fix this error by setting user.password= user.password.encode('utf8'), I get "Invalid Salt".

What is wrong with this code?

UPDATE: I am storing the passwords through Flask input from the user:

import json
import sys
import os
import plotly
import pandas as pd
import numpy as np
import plotly.graph_objs as go


from scripts import tabledef
from scripts import forms
from scripts import helpers
from flask import Flask, redirect, url_for, render_template, request, session, flash, Markup
from flask_socketio import SocketIO, emit

@app.route('/', methods=['GET', 'POST'])
def login():
    if not session.get('logged_in'):
        form = forms.LoginForm(request.form)
        if request.method == 'POST':
            username = request.form['username'].lower()
            password = request.form['password']
            if form.validate():
                if helpers.credentials_valid(username, password):
                    session['logged_in'] = True
                    session['username'] = username
                    session['email'] = request.form['email']
                    session['password'] = request.form['password']
                    return json.dumps({'status': 'Login successful'})
                return json.dumps({'status': 'Invalid user/pass'})
            return json.dumps({'status': 'Both fields required'})
        return render_template('login.html', form=form)
    user = helpers.get_user()
    return render_template('home.html', user=user)

@app.route('/signup', methods=['GET', 'POST'])
def signup():
    if not session.get('logged_in'):
        form = forms.LoginForm(request.form)
        if request.method == 'POST':
            username = request.form['username'].lower()
            password = helpers.hash_password(request.form['password'])
            email = request.form['email']
            if form.validate():
                if not helpers.username_taken(username):
                    helpers.add_user(username, password, email)
                    session['logged_in'] = True
                    session['username'] = username
                    session['email'] = request.form['email']
                    session['password'] = request.form['password']
                    return json.dumps({'status': 'Signup successful'})
                return json.dumps({'status': 'Username taken'})
            return json.dumps({'status': 'User/Pass required'})
        return render_template('login.html', form=form)
    return redirect(url_for('login'))

This is the error I get:

/lib/python3.5/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/suraj/Documents/Programming/current-projects/GW_Dining_Tracker/env/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/suraj/Documents/Programming/current-projects/GW_Dining_Tracker/env/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/suraj/Documents/Programming/current-projects/GW_Dining_Tracker/env/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/suraj/Documents/Programming/current-projects/GW_Dining_Tracker/Flaskex-master/app.py", line 34, in login
    if helpers.credentials_valid(username, password):
  File "/home/suraj/Documents/Programming/current-projects/GW_Dining_Tracker/Flaskex-master/scripts/helpers.py", line 64, in credentials_valid
    return bcrypt.checkpw(password.encode('utf8'), user.password)
  File "/home/suraj/Documents/Programming/current-projects/GW_Dining_Tracker/env/lib/python3.5/site-packages/bcrypt/__init__.py", line 101, in checkpw
    raise TypeError("Unicode-objects must be encoded before checking")
TypeError: Unicode-objects must be encoded before checking

Answer 1

The problem is that you're taking the value from a SQLAlchemy String column and passing it to bcrypt.checkpw. String is intended for Unicode strings, it provides values as str. But bcrypt only works on byte strings, so it expects a bytes. That's what the TypeError that says "Unicode-objects must be encoded before hashing" is telling you.

Depending on what database backend and DB-API library you're using (and, for some backends, on how your database is configured), when you save a bytes value s to a String column, it might save s.decode(), in which case you could just use user.password.encode() to get the same bytes back—but it might not. For example, it could also just save, say, str(s). In which case, if the hash were the bytes value b'abcd', the column value would be the string "b'abcd'", so and calling encode on that gets you b"b'abcd'", not b'abcd'.

The cleanest way to handle this is to use a Binary column¹—or, maybe better, Binary(60)²—to store your hashes, instead of a String column. Any DB-API that supports Binary will just store a bytes as-is, and return it as a bytes, which is exactly what you want.

---

_{1. Binary is an optional type. If it isn't present for your DB-ABI, the same type may be available as BINARY. If not, look through the list of types and try other types that inherit from _Binary. The ones without "large" in their name or acronym will probably be more efficient, but otherwise any of them should work.}

_{2. With the default settings, bcrypt printable digests will always be exactly 60 bytes. Databases can generally store fixed-width fields like BINARY(60) more compactly, and search them more quickly than variable-width fields like VARBINARY. Just using plain BINARY may be fine, but it may also work like VARBINARY, or it may waste space and work like BINARY(255), etc.}