7 October, 2011 by webjunkie on Postgres, Pylons, Python, SQLAlchemy, UUID, uWSGI

UUID Objects in Pylons with SQLAlchemy

Recently I had to generate and store uuid objects into a Postgres database, but to my surprise SQLAlchemy kept showing the following error:

sqlalchemy.exc.ProgrammingError: (ProgrammingError) can't adapt type 'UUID'

So the solution was that I had to change my field type from varchar to uuid on my Postgres database, import the psycopg2 extras functions and register the uuid type:

import psycopg2.extras
import sqlalchemy
from sqlalchemy import create_engine
from sqlalchemy import MetaData,Column, Table, ForeignKey
engine = create_engine('postgresql://user:pass@localhost/db',echo=True)
metadata = MetaData(bind=engine)
hash_key = uuid.uuid4()
psycopg2.extras.register_uuid()
conn = engine.connect()
query = conn.execute("insert into tbl (uuid_col) values(%(uuid)s",uuid=hash_key)

And voila, values were inserted correctly on my database.

7 July, 2011 by webjunkie on Facebook, Oauth, Postgres, Pylons, Python, SQLAlchemy

Pylons and Facebook based Authorization with OAuth 2.0

Authorizing a user using Facebook OAuth 2.0 based login it’s pretty straighfoard. This is for the person who asked a question on stackoverflow.com I hope it helps.

Add the apikey, appid and secret of my Facebook app to the development.ini file

facebook.apikey = 42bc5a4051b274ede5cb73a6cd6fad56
facebook.secret = a355e66e8c0293390896a170f9d5r4c3
facebook.appid = 20894420580890

Configure my model/init.py file:

from PRJNAME.model.meta import Session, Base
from sqlalchemy import Column
from sqlalchemy import Integer
from sqlalchemy import Unicode
from sqlalchemy import Table
from sqlalchemy import Text
from sqlalchemy import Unicode
from sqlalchemy import BigInteger
from sqlalchemy import TIMESTAMP
from sqlalchemy import ForeignKey,ForeignKeyConstraint
from sqlalchemy.orm import mapper
import datetime
def init_model(engine):
    """Call me before using any of the tables or classes in the model"""
    Session.configure(bind=engine,autoflush=True,autocommit=False)
def now():
    return datetime.datetime.now()
social_user = Table('social_user', Base.metadata,
    Column('id', Integer, primary_key=True),
    Column('oauth_provider', Text(), nullable=False),
    Column('oauth_id', Text() , nullable=False),
    Column('username', Unicode(50), nullable=False),
    Column('access_token', Unicode(200)),
    Column('created', TIMESTAMP(timezone=True), default=now()),
    )
  class Social(object):
    pass
  mapper(Social,social_user)

Download the Facebook Python-SDK

https://github.com/facebook/python-sdk

And save it on your project lib directory

Create a controller named account

~$ paster controller account

Add a view named login and render a login button

I’ll use https://graph.facebook.com/oauth/authorize as the url to start the authorization process

 import PRJNAME.lib.facebook as facebook
 import simplejson
 from pylons import config
 class AccountController(BaseController):
   def login(self):
     params = {
       'client_id':config['facebook.appid'],
       'scope' : 'offline_access,publish_stream,email', #permissions needed by the app
       'redirect_uri' : 'http://'+request.environ['HTTP_HOST']+'/account/auth'
     }
     keys = params.keys()[:]
     keys.sort()
     q_str = unicode()
     for name in keys:
       q_str+= ("&%s=%s" % (name,params[name]))
     c.login_url = 'https://graph.facebook.com/oauth/authorize?' + q_str
     return render('/website/login.html')

login.html

Login with Facebook

Add a function on helpers.py

This function will create our access_token from our code request params

  import urllib
  import json
  def getAccessTokenFromCode(applicationId, applicationSecret, code,redirect_uri):
    q_str = unicode()
    params = {
      'client_id':applicationId,
      'client_secret':applicationSecret,
      'code':code,
      'redirect_uri':'http://'+request.environ['HTTP_HOST']+redirect_uri
    }
    keys = params.keys()[:]
    keys.sort()
    for name in keys:
      q_str+= ("&%s=%s" % (name, params[name]))
    url = 'https://graph.facebook.com/oauth/access_token?'+ q_str
    response = urllib.urlopen(url).read()
    result = response.split('=')[-1]
    if result:
      return result

Create an auth view on account controller

I do this because in the link button there is an argument called redirect_uri that points to http://ww.example.com/account/auth.

from PRJNAME.lib.helpers import getAccessTokenFromCode
from pylons import config
def auth(self):
if request.params.has_key('code'):
      access_token = getAccessTokenFromCode(
       config['facebook.appid'],
       config['facebook.secret'],
       request.params.get('code'),
       redirect_uri='/account/auth'
      )
      graph = facebook.GraphAPI(access_token)
      #grab the user data
      fbdata = graph.get_object("me")
      #query the db to check if user is already authorized
      q = Session.query(model.Social)
      user = q.filter_by(oauth_id=fbdata['id']).first()
      if user:
       # you can redirect to a dashboard
       return 'already in db and authorized'
      else:
      #insert user data to db
      fb = model.Social()
      fb.oauth_provider = 'Facebook'
      fb.oauth_id = fbdata['id']
      fb.username = fbdata['name']
      fb.access_token = access_token
      Session.add(fb)
      Session.commit()
      # redirect to dashboard page

And that’s pretty much it.

29 May, 2011 by webjunkie on DB, Debian, Dovecot, Postfix, Postgres, uWSGI

Host Email with Postfix, Dovecot and Postgres on Debian 6 (Squeeze)

Recently I had to configure a web server from scratch, which I haven’t done in a while. So I took a look at the guides from my current hosting provider Linode. They have a guide covering debian 5 (Lenny) which is basically the same for debian 6 (squeeze). To make this a bit more exciting I decided to replace MySQL for Postgres since I really didn’t need it. So I will only post about the changes you need to perform based from the Linode guide.

Install Required Packages

Replace the mysql related packages for postgres and replace mailx for bsd-mailx.

apt-get install postfix postfix-pgsql postfix-doc dovecot-common dovecot-imapd dovecot-pop3d postfix-tls libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-pgsql openssl telnet bsd-mailx postgresql-contrib postgresql-8.4

Set up Postgres for Virtual Domains and Users

Open /etc/postgres/8.4/main/pg_hba.conf and make sure local connections require a password

host    all         all         127.0.0.1/32          md5

You could also make all local connections trusted and in this case no password will be needed, but of course it could represent a security risk

host    all         all         127.0.0.1/32          trust

This next step is optional but as a security measure I like to have Postgres listen on a diferent port other than the default.
Open /etc/postgres/8.4/main/postgres.conf and uncomment the following line and add 6789 as the port number

port = 6789

Create the user for the mail database.

sudo -u postgres createuser  -P -E -s -d mail_admin

Now create the database.

sudo -u postgres createdb -O mail_admin mail -E UTF-8

Grant all privileges.

GRANT ALL PRIVILEGES ON DATABASE mail to mail_admin;

Create domains, forwardings, users and transport tables.

CREATE TABLE domains (domain varchar(50) NOT NULL, PRIMARY KEY (domain) );
CREATE TABLE forwardings (source varchar(80) NOT NULL, destination TEXT NOT NULL, PRIMARY KEY (source) );
CREATE TABLE users (email varchar(80) NOT NULL, password varchar(20) NOT NULL, PRIMARY KEY (email) );
CREATE TABLE transport ( domain varchar(128) NOT NULL default '', transport varchar(128) NOT NULL default '', constraint fieldomainunique UNIQUE  (domain) );

Now I’ll create a domain and user in mail database. But first I will use Postgres pgcrypto functions since passwords need to be encrypted using the UNIX original encryption scheme. Replace your_password with your own password.

mail=#\i /usr/share/postgresql/8.4/contrib/pgcrypto.sql

INSERT INTO domains (domain) VALUES ('example.com');

INSERT INTO users (email, password) VALUES ('info@example.com', crypt('your_password', gen_salt('des') ));

mail=#\q

Configure Postfix to work with Postgres

Create file /etc/postfix/pgsql-virtual_domains.cf

user = mail_admin
password = mail_admin_password
dbname = mail
query = SELECT domain AS virtual FROM domains WHERE domain='%s'
hosts = 127.0.0.1:6789

Create file /etc/postfix/pgsql-virtual_forwardings.cf

user = mail_admin
password = mail_admin_password
dbname = mail
query = SELECT destination FROM forwardings WHERE source='%s'
hosts = 127.0.0.1:6789

Create file /etc/postfix/pgsql-virtual_mailboxes.cf

user = mail_admin
password = mail_admin_password
dbname = mail
query = SELECT split_part(email, '@', 2) || '/' || split_part(email,'@', 1) FROM users WHERE email='%s';
hosts = 127.0.0.1:6789

Create file /etc/postfix/pgsql-virtual_email2email.cf

user = mail_admin
password = mail_admin_password
dbname = mail
query = SELECT email FROM users WHERE email='%s'
hosts = 127.0.0.1:6789

Set the proper permissions

chmod o= /etc/postfix/pgsql-virtual_*.cf
chgrp postfix /etc/postfix/pgsql-virtual_*.cf

Add the following one line at a time. Replace qualified.hostname.com with your hostname

postconf -e 'myhostname = qualified.hostname.com'
postconf -e 'mydestination = qualified.hostname.com, localhost, localhost.localdomain'
postconf -e 'mynetworks = 127.0.0.0/8'
postconf -e 'message_size_limit = 30720000'
postconf -e 'virtual_alias_domains ='
postconf -e 'virtual_alias_maps = proxy:pgsql:/etc/postfix/pgsql-virtual_forwardings.cf, pgsql:/etc/postfix/pgsql-virtual_email2email.cf'
postconf -e 'virtual_mailbox_domains = proxy:pgsql:/etc/postfix/pgsql-virtual_domains.cf'
postconf -e 'virtual_mailbox_maps = proxy:pgsql:/etc/postfix/pgsql-virtual_mailboxes.cf'
postconf -e 'virtual_mailbox_base = /home/vmail'postconf -e 'virtual_uid_maps = static:5000'postconf -e 'virtual_gid_maps = static:5000'
postconf -e 'smtpd_sasl_auth_enable = yes'postconf -e 'broken_sasl_auth_clients = yes'
postconf -e 'smtpd_sasl_authenticated_header = yes'
postconf -e 'smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination'
postconf -e 'smtpd_use_tls = yes'
postconf -e 'smtpd_tls_cert_file = /etc/postfix/smtpd.cert'
postconf -e 'smtpd_tls_key_file = /etc/postfix/smtpd.key'
postconf -e 'virtual_create_maildirsize = yes'
postconf -e 'virtual_maildir_extended = yes'
postconf -e 'proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $virtual_mailbox_limit_maps'
postconf -e virtual_transport=dovecotpostconf -e dovecot_destination_recipient_limit=1

Change postfix listening port to 2525 in /etc/postfix/master.cf.

2525      inet  n       -       -       -       -       smtpd

Or you could just uncomment the submission line and default to port 587.

submission inet n       -       -       -       -       smtpd

Configure saslauthd to use Postgres

Create file /etc/pam.d/smtp. Replace your_password with your own password.

auth    required   pam_pgsql.so user=mail_admin passwd=your_password host=127.0.0.1 port=6789 db=mail table=users usercolumn=email passwdcolumn=password crypt=1
account sufficient pam_pgsql.so user=mail_admin passwd=your_password host=127.0.0.1 port=6789 db=mail table=users usercolumn=email passwdcolumn=password crypt=1

Create file /etc/postfix/sasl/smtpd.conf. Replace your_password with your own password.

pwcheck_method: saslauthd
mech_list: plain login
allow_plaintext: true
auxprop_plugin: sql
sql_engine: pgsql
sql_hostnames: 127.0.0.1:6789
sql_user: mail_admin
sql_passwd: your_password
sql_database: mail
sql_select: select password from users where email = '%u'

Create file /etc/pam_pgsql.conf. Replace your_password with your own password.

database = mail
host = localhost
user = mail_admin
password = your_password
table = users
user_column = email
pwd_column = password
pw_type = crypt

Set proper permissions to files.

chmod o= /etc/pam_pgsql.conf
chmod o= /etc/postfix/sasl/smtpd.conf

Configure Dovecot

One thing that was pointed out at the end of the guide by user Nathan is to change cmusieve for sieve on file /etc/dovecot/dovecot.conf on this following part

protocol lda {
    log_path = /home/vmail/dovecot-deliver.log
    auth_socket_path = /var/run/dovecot/auth-master
    postmaster_address = postmaster@example.com
    mail_plugins = cmusieve #change to sieve
    global_script_path = /home/vmail/globalsieverc
}

Create file /etc/dovecot/dovecot-sql.conf. Replace your_password with your own password.

driver = pgsql
connect = host=127.0.0.1 port=6789 dbname=mail user=mail_admin password=your_password
default_pass_scheme = CRYPT
password_query = SELECT email as user, password FROM users WHERE email='%u';

And thats about it. If you managed to configure everything correctly congratulations!, if not post a comment and we’ll try to find the solution.

Troubleshooting

Remember that /var/log/syslog and /var/log/mail.log are your friends. You can also add the word debug at the end of each line on /etc/pam.d/smtp to help you troubleshoot any problems you may encounter.

Guillermo Cruz

Fullstack Developer

Postgres