07 3 / 2014

Python has `sorted` function which sorts iterable in ascending or descending order.

``````# Sort descending
In [95]: sorted([1, 2, 3, 4], reverse=True)
Out[95]: [4, 3, 2, 1]

# Sort ascending
In [96]: sorted([1, 2, 3, 4], reverse=False)
Out[96]: [1, 2, 3, 4]
``````

`sorted(iterable, reverse=True)[:n]` will yield first `n` largest numbers. There is an alternate way.

Python has heapq which implements heap datastructure. `heapq` has function `nlargest` and `nsmallest` which take arguments `n` number of elements, `iterable like list, dict, tuple, generator` and optional argument `key`.

``````In [85]: heapq.nlargest(10, [1, 2, 3, 4,])
Out[85]: [4, 3, 2, 1]

In [88]: heapq.nlargest(10, xrange(1000))
Out[88]: [999, 998, 997, 996, 995, 994, 993, 992, 991, 990]

In [89]: heapq.nlargest(10, [1000]*10)
Out[89]: [1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]

In [99]: heapq.nsmallest(3, [-10, -10.0, 20.34, 0.34, 1])
Out[99]: [-10, -10.0, 0.34]
``````

Let’s say `marks` is a list of dictionary containing students marks. Now with `heapq` it is possible to find highest and lowest mark in a subject.

``````In [113]: marks = [{'name': "Ram", 'chemistry': 23},{'name': 'Kumar', 'chemistry': 50}, {'name': 'Franklin', 'chemistry': 89}]

In [114]: heapq.nlargest(1, marks, key=lambda mark: mark['chemistry'])
Out[114]: [{'chemistry': 89, 'name': 'Franklin'}]

In [115]: heapq.nsmallest(1, marks, key=lambda mark: mark['chemistry'])
Out[115]: [{'chemistry': 23, 'name': 'Ram'}]
``````

`heapq` can be used for building priority queue.

Note: IPython is used in examples where `In [114]:` means `Input line number 114` and `Out[114]` means `Output line number 114`.

Tags:

27 2 / 2014

Let’s say you want to find how many times each element is present in the list or tuple.

#### Normal approach

``````words = ['a', 'the', 'an', 'a', 'an', 'the']
d = {}
for word in words:
if word in d:
d[word] += 1
else:
d[word] = 1
print d
{'a': 2, 'the': 2, 'an': 2}
``````

#### Better approach

``````words = ['a', 'the', 'an', 'a', 'an', 'the']
d = {}
for word in words:
d[word] = d.get(word, 0) + 1

print d
{'a': 2, 'the': 2, 'an': 2
``````

Both the approach returned same values. The first one has 6 lines of logic and second has 3 lines of logic (less code less management).

Second approach uses `d.get` method. `d.get(word, 0)` return count of the word if key is present else `0`. If `0` isn’t passed `get` will return `None`.

#### Pythonic approach:

``````import collections

words = ['a', 'b', 'a']

res = collections.Counter(words)

print res
Counter({'a': 2, 'b': 1})
``````

Last approach is just one line and pythonic.

Snippet is extracted from Transforming Code into Beautiful, Idiomatic Python. Do watch and enjoy.

Tags:

08 2 / 2014

It is very common to update single attribute of a model instance (say update `first name` in user profile) and save it to db.

``````In [18]: u = User.objects.get(id=1)

In [19]: u.first_name = u"kracekumar"

In [20]: u.save()
``````

Very straight forward approach. How does django send the sql query to database ?

``````In [22]: from django.db import connection

In [22]: connection.queries
Out[22]:
[...
{u'sql': u'UPDATE "auth_user" SET "password" = \'pbkdf2_sha256\$12000\$vsHWOlo1ZhZg\$DrC46wq+a2jEtEzxmUEw4vQw8oV/rxEK7zVi30QLGF4=\', "last_login" = \'2014-02-01 06:55:44.741284+00:00\', "is_superuser" = true, "username" = \'kracekumar\', "first_name" = \'kracekumar\', "last_name" = \'\', "email" = \'me@kracekumar.com\', "is_staff" = true, "is_active" = true, "date_joined" = \'2014-01-30 18:41:18.174353+00:00\' WHERE "auth_user"."id" = 1 ', u'time': u'0.001'}]
``````

Not happy. Honestly it should be `UPDATE auth_user SET first_name = 'kracekumar' WHERE id = 1`. Django should ideally update modified fields.

Right way to do is

``````In [23]: User.objects.filter(id=u.id).update(first_name="kracekumar")
Out[23]: 1

In [24]: connection.queries
Out[24]:
[...
{u'sql': u'UPDATE "auth_user" SET "first_name" = \'kracekumar\' WHERE "auth_user"."id" = 1 ', u'time': u'0.001'}]
``````

Yay! Though both queries took same amount of time, latter is better.

### Edit: There is one more cleaner way to do it.

``````In [60]: u.save(update_fields=['first_name'])

In [61]: connection.queries
Out[61]:
[...
{u'sql': u'UPDATE "auth_user" SET "first_name" = \'kracekumar\'  WHERE "auth_user"."id" = 1 ',
u'time': u'0.001'}]
``````

Tags:

26 12 / 2013

This is the material which I use for teaching python to beginners.

tld;dr: Very minimal explanation more code.

Python?

• Interpreted language

Introduction

``````hasgeek@hasgeek-MacBook:~/codes/python/hacknight\$ python
Python 2.7.3 (default, Aug  1 2012, 05:14:39)
[GCC 4.6.3] on linux2
>>>

>>> print "Let's learn Python"
Let's learn Python
``````

Numbers

``````>>> 23 + 43
66
>>> 23 - 45
-22
>>> 23 * 45
1035
>>> 23 ** 4
279841
>>> 23 / 4
5
>>> 23 / 4.0
5.75
>>> 7 % 2
1
``````

Expressions

``````>>> 3 < 2
False
>>> 3 > 2
True
>>> 3 > 2 < 1
False
>>> (3 > 2) and (2 < 1)
False
>>> 3 > 2 > 1 > 0
True
>>> (3 > 2) and (2 > 1) and (1 > 0)
True
>>> 1 or 2
1
>>> 2 or 1
2
>>> 1 + 2 + 3 * 4 + 5
20
1 + 2 + 3 * 4 + 5
↓
3   + 3 * 4 + 5
↓
3   +   12  + 5
↓
15      + 5
↓
20

>>> "python" > "perl"
True
>>> "python" > "java"
True
``````

Variables

``````>>> a = 23
>>> print a
23
>>> a = "Python"
>>> print a
Python
``````

Guess the output

``````True = False
False = True
print True, False
print 2 > 3
``````

Parallel Assignment

``````>>> language, version = "Python", 2.7
>>> print language, version
Python 2.7
>>> x = 23
>>> x = 23
>>> y = 20
>>> x, y = x, x + y
>>> print x, y
23 43
``````

Guess the output

``````z, y = 23, z + 23
a, b = 23, 12, 20
a = 1, 2
``````

Swap Variable

``````>>> x = 12
>>> y = 21
>>> x, y = y, x
>>> print x, y
21 12
>>>
``````

String

``````>>> language = "Python"
>>> print language
Python
>>> language = 'Python'
>>> print language
Python
>>> language = """Python"""
>>> print language
Python
>>> description = """Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability.
... It is an expressive language which provides language constructs intended to enable clear programs on both a small and large scale.
... Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles.
... """
>>> print description
Python is a general-purpose, high-level programming language whose design philosophy emphasizes code readability.
It is an expressive language which provides language constructs intended to enable clear programs on both a small and large scale.
Python supports multiple programming paradigms, including object-oriented, imperative and functional programming styles.
>>>
``````

Guess output

``````name = "krace" + "kumar"
print name
print name[0]
name[0] = "K"
``````

Guess output

``````print 1 + 2.5
print "kracekumar" + 23
``````

Condition

Write a program to find greatest of two numbers.

``````>>> a = 12
>>> b = 23
>>> if a > b:
...     print "a is greater than b"
... else:
...     print "b is greater than a"
...
b is greater than a
>>> if a > 0:
...     print "a is positive"
... elif a == 0:
...     print "a is zero"
... elif a < 0:
...     print "a is negative"
...
a is positive
``````

Data Structure

List

List is a collection of heterogenous data types like integer, float, string.

``````>>> a = [1, 2, 3]
>>> b = ["Python", 2.73, 3]
>>> len(a)
3
>>> len(b)
3
>>> a[0]
1
>>> a[-1]
3
>>> b[2]
3
>>> [1, 2] + [3, 4]
[1, 2, 3, 4]
>>> all = [a, b]
>>> all[0]
[1, 2, 3]
>>> all[-1]
['Python', 2.73, 3]
>>> all[3]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> all.append("Bangalore")
>>> all
[[1, 2, 3], ['Python', 2.73, 3], 'Bangalore']
>>> del all[-1]
>>> all
[[1, 2, 3], ['Python', 2.73, 3]]
>>> all[1] = "insert"
>>> all
[[1, 2, 3], 'insert']
>>> all
[[1, 2, 3], 'insert']
>>> 'insert' in all
True
>>> range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> range(10, 2)
[]
>>> range(10, 0, -1)
[10, 9, 8, 7, 6, 5, 4, 3, 2, 1]
>>> range(0, 12, 1)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

range() -> `range([start,] stop[, step]) -> list of integers`
``````

Slicing

``````>>> l = [1, 2, 3, 4, 5, 6, 7]
[1, 2, 3, 4, 5, 6, 7]
>>> l[:2] #first two elements
[1, 2]
>>> l[2:] #exclude first two elements
[3, 4, 5, 6, 7]
>>> l[::2] #every second element
[1, 3, 5, 7]
>>> l[::1] #every element
[1, 2, 3, 4, 5, 6, 7]
>>> l[::3] #every third element
[1, 4, 7]
>>> l[::10] #every tenth element
[1]
>>> l[::-1]
[7, 6, 5, 4, 3, 2, 1]
``````

Guess the output

l[1:7:2] [][:2] [1][:2]

Accessing list elements

``````>>> for item in all:
...     print item
...
[1, 2, 3]
insert
>>> for number in range(10):
...     print number
...
0
1
2
3
4
5
6
7
8
9
``````

Find all odd numbers from 0 to 9

``````>>> for number in range(0, 10):
...     if number % 2:
...         print number
...
1
3
5
7
9
``````

inbuilt functions

``````>>> help([])
>>> min([1, 2, 3])
1
>>> max([1, 2, 3])
3
>>> sum([1, 2, 3])
6
>>> pow(2, 3)
8
``````

Write program which takes a number as input and if number is divisible by 3, 5 print Fizz, Buzz, FizzBuzz respectively

``````import sys

if __name__ == "__main__":
if len(sys.argv) == 2:
number = int(sys.argv[1])
if number % 15 == 0:
print "FizzBuzz"
elif number % 3 == 0:
print "Fizz"
elif number % 5 == 0:
print "Buzz"
else:
print number

else:
print "python filename.py 23 is the format"
``````

Tuples

Tuple is a sequence type just like list, but it is immutable. A tuple consists of a number of values separated by commas.

``````>>> t = (1, 2)
>>> t
(1, 2)
>>> t[0]
1
>>> t[0] = 1.1
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object does not support item assignment
>>> t = 1, 2
>>> t
(1, 2)
>>> del t[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'tuple' object doesn't support item deletion
>>> for item in t:
...     print item
...
1
2
``````

Sets

Sets are unordered collection of unique elements.

``````>>> x = set([1, 2, 1])
>>> x
set([1, 2])
>>> x
set([1, 2, 3])
>>> x = {1, 3, 4, 1}
>>> x
set([1, 3, 4])
>>> 1 in x
True
>>> -1 in x
False
>>>
``````

Again Lists

``````>>> even_numbers = []
>>> for number in range(0, 9):
...     if number % 2 == 0:
...         even_numbers.append(number)
...
>>> even_numbers
[0, 2, 4, 6, 8]
``````

`As a programmer your job is write lesser code`

List Comprehensions

``````>>> [x for x in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [x + 1 for x in range(10)]
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> numbers = []
>>> for x in range(10):
...     numbers.append(x + 1)
...
>>> print numbers
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> even_numbers = [x for x in range(10) if x %2 == 0]
>>> even_numbers
[0, 2, 4, 6, 8]
>>> [(x, y) for x in range(5) for y in range(5) if (x+y)%2 == 0]
[(0, 0), (0, 2), (0, 4), (1, 1), (1, 3), (2, 0), (2, 2), (2, 4), (3, 1), (3, 3    ), (4, 0), (4, 2), (4, 4)]
>>>
``````

Dictionaries

``````>>> d = {'a': 1, 'b': 2, 'c': 3}
>>> d['a']
1
>>> d.get('a')
1
>>> d['z']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'z'
>>> d.get('z')
>>>
>>> d['a'] = 2
>>> d
{'a': 2, 'c': 3, 'b': 2}
>>> d['z'] = 26
>>> d
{'a': 2, 'c': 3, 'b': 2, 'z': 26}
>>> d.keys()
['a', 'c', 'b', 'z']
>>> d.values()
[2, 3, 2, 26]
>>> d.items()
[('a', 2), ('c', 3), ('b', 2), ('z', 26)]
>>> type(d.items())
<type 'list'>
>>> d = {'a': 2, 'b': 2, 'c': 3, 'z': 26}
>>> for key in d:
...     print key
...
a
c
b
z
>>> for key, value in d.items():
...     print key, value
...
a 2
c 3
b 2
z 26
>>> 'a' in d
True
>>> d.has_key('a')
True
``````

Function

Just like a value can be associated with a name, a piece of logic can also be associated with a name by defining a function.

``````>>> def square(x):
...     return x * x
...
>>> square(2)
4
>>> square(2+1)
9
>>> square(x=5)
25
>>> def dont_return(name):
...     print "Master %s ordered not to return value" % name
...
>>> dont_return("Python")
Master Python ordered not to return value
>>> def power(base, to_raise=2):
...     return base ** to_raise
...
>>> power(3)
9
>>> power(3, 3)
27
>>> def power(to_raise=2, base):
...     return base ** to_raise
...
File "<stdin>", line 1
SyntaxError: non-default argument follows default argument
>>> square(3) + square(4)
25
>>> power(base=square(2))
16
>>> def sum_of_square(x, y):
...     return square(x) + square(y)
...
>>> sum_of_square(2, 3)
13
>>> s = square
>>> s(4)
16
>>> def fxy(f, x, y):
...     return f(x) + f(y)
...
>>> fxy(square, 3, 4)
25
``````

Methods - Methods are special kind of functions that work on an object.

``````>>> lang = "Python"
>>> type(lang)
<type 'str'>
>>> dir(lang)
['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
'__getitem__', '__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__', '__le__', '__len__', '__lt__',
'__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__', '_formatter_field_name_split', '_formatter_parser',
'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'format', 'index', 'isalnum',
'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'partition',
'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip',     'split', 'splitlines',
'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
>>> lang.upper()
'PYTHON'
>>> help(lang.upper)
>>> lang.startswith('P')
True
>>> help(lang.startswith)

>>> lang.startswith('y', 1)
True
``````

Files

``````>>> f = open('foo.txt', 'w')
>>> help(f)

>>> f.write("First line")
>>> f.close()
>>> f = open('foo.txt', 'r')
'First line'
''
>>> f = open('foo.txt', 'a')
>>> f.write('Second line')
>>> f.close()
>>> f = open('foo.txt', 'r')
'First lineSecond line'
>>> f = open('foo.txt', 'a')
>>> f.write("New line\n")
>>> f.write("One more new line")
>>> f.close()
>>> f = open('foo.txt', 'r')
'First lineSecond lineNew line\n'
'One more new line'
''
>>> f.close()
>>> f = open('foo.txt')
['First lineSecond lineNew line\n', 'One more new line']
>>> f = open('foo.txt', 'w')
>>> f.writelines(["1\n", "2\n"])
>>> f.close()
>>> f = open('foo.txt')
['1\n', '2\n']
>>> f.close()
``````

Exception Handling

``````>>> f = open('a.txt')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 2] No such file or directory: 'a.txt'
>>> try:
...     f = open('a.txt')
... except:
...     print "Exception occured"
...
Exception occured
>>> try:
...     f = open('a.txt')
... except IOError, e:
...     print e.message
...

>>> e
IOError(2, 'No such file or directory')
>>> dir(e)
['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__getslice__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__unicode__', 'args', 'errno',     'filename', 'message', 'strerror']
>>> e.strerror
'No such file or directory'
>>> try:
...     print l[4]
... except IndexError, e:
...     print e
...
list index out of range
>>> raise Exception("error message")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Exception: error message
>>> try:
...     print "a"
...     raise Exception("doom")
... except:
...     print "b"
... else:
...     print "c"
... finally:
...     print "d"
...
a
b
d
``````

Object Oriented Programming

``````>>> class BankAccount:
def __init__(self):
self.balance = 0

def withdraw(self, amount):
self.balance -= amount
return self.balance

def deposit(self, amount):
self.balance += amount
return self.balance

>>> a = BankAccount()
>>> b = BankAccount()
>>> a.deposit(200)
200
>>> b.deposit(500)
500
>>> a.withdraw(20)
180
>>> b.withdraw(1000)
-500
>>> class MinimumBalanceAccount(BankAccount):
...    def __init__(self, minimum_balance):
...        BankAccount.__init__(self)
...        self.minimum_balance = minimum_balance
...
...    def withdraw(self, amount):
...        if self.balance - amount < self.minimum_balance:
...            print "Sorry, you need to maintain minimum balance"
...        else:
...            return BankAccount.withdraw(self, amount)
>>> a = MinimumBalanceAccount(500)
>>> a
<__main__.MinimumBalanceAccount instance at 0x7fa0bf329878>
>>> a.deposit(2000)
2000
>>> a.withdraw(1000)
1000
>>> a.withdraw(1000)
Sorry, you need to maintain minimum balance
>>> class A:
...     def f(self):
...         return self.g()
...     def g(self):
...         return "A"
...
>>> a = A()
>>> a.f()
'A'
>>> a.g()
'A'
>>> class A:
...     def __init__(self):
...         self._protected = 1
...         self.__private = 2
...
>>> a = A()
>>> a._protected
1
>>> a.__private
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute '__private'
``````

Sample Python Program

``````#! /usr/bin/env python
# -*- coding: utf-8 -*-

class BankAccount:
def __init__(self):
self.balance = 0

def withdraw(self, amount):
self.balance -= amount
return self.balance

def deposit(self, amount):
self.balance += amount
return self.balance

class MinimumBalanceAccount(BankAccount):
def __init__(self, minimum_balance):
BankAccount.__init__(self)
self.minimum_balance = minimum_balance

def withdraw(self, amount):
if self.balance - amount < self.minimum_balance:
print "Sorry, you need to maintain minimum balance"
else:
return BankAccount.withdraw(self, amount)

def __repr__(self):
return "MinimuBalanceAccount, Balance: %d" %(self.balance)

if __name__ == "__main__":
a = MinimumBalanceAccount(500)
print a.deposit(5000)
print a.withdraw(4500)
print a.withdraw(500)
``````

Few examples are taken from python practice book.

Github repo: https://github.com/kracekumar/python-training

Tags:

25 12 / 2013

This article will focus on deploying flask app starting from scratch like creating separate linux user, installating database, web server. Web server will be nginx, database will be postgres, python 2.7 middleware will be uwsgi, server ubuntu 13.10 x64. Flask app name is `fido`. Demo is carried out in Digital ocean.

### Step 1 - Installation

``````root@fido:~# apt-get install -y build-essential python-dev
``````

Install uwsgi dependencies

``````root@fido:~# apt-get install -y libxml2-dev libxslt1-dev
``````

Nginx, uwsgi

``````root@fido:~#  apt-get install -y nginx uwsgi uwsgi-plugin-python
``````

Start nginx

``````root@fido:~# service nginx start
* Starting nginx nginx                                                          [ OK ]
``````

Postgres

``````root@fido:~# apt-get install -y postgresql postgresql-contrib libpq-dev
``````

### Step 2 - User

Create a new linux user fido

``````root@fido:~# adduser fido
``````

Enter all the required details.

``````root@fido:~# ls /home
fido
``````

Successfully new user is created.

Grant `fido` root privilege.

``````root@fido:~# /usr/sbin/visudo
# User privilege specification

root    ALL=(ALL:ALL) ALL
fido    ALL=(ALL:ALL) ALL
``````

Since `fido` is not normal user, delete fido’s home directory.

``````root@fido:~# rm -rf /home/fido
root@fido:~# ls /home
root@fido:~#
``````

Create a new db user fido

``````root@fido:~# su - postgres
postgres@fido:~\$ createuser --pwprompt
Enter name of role to add: fido
Enter it again:
Shall the new role be a superuser? (y/n) y
``````

`--pwprompt` will prompt for password. `release` is the password I typed (we need this to connect db from app).

Create a new database fido

``````postgres@fido:~\$ createdb fido;
postgres@fido:~\$ psql -U fido -h localhost
psql (9.1.10)
SSL connection (cipher: DHE-RSA-AES256-SHA, bits: 256)
Type "help" for help.
fido=# \d
No relations found.
``````

Done. New database role `fido` and database is created. We are successfully able to login.

### Step 3 - Python dependencies

Install pip

``````root@fido:# cd /tmp
root@fido:/tmp# wget https://bitbucket.org/pypa/setuptools/raw/bootstrap/ez_setup.py
root@fido:/tmp# python ez_setup.py install
root@fido:/tmp# easy_install pip
# What is easy_install ? Python package manager.
# what is pip ? Python package manager.
# How to install pip ? easy_install pip.
# Shame on python :-(
...
Installed /usr/local/lib/python2.7/dist-packages/pip-1.4.1-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip
``````

Install virtualenv

``````root@fido:/tmp# pip install virtualenv
``````

### Step 4 - Install app dependencies

Here is the sample app code. The app is just for demo. The app will be placed in `/var/www/fido`. Normally in production, this will be similar to `git clone <url>` or `hg clone <url>` inside directory. Make sure you aren’t sudo while cloning.

``````root@fido:/tmp# cd /var
root@fido:/var# mkdir www
root@fido:/var# mkdir www/fido
``````

Change the owner of the repo to `fido`.

``````root@fido:/var# chown fido:fido www/fido
root@fido:/var# ls -la www/
total 12
drwxr-xr-x  3 root root 4096 Dec 25 03:18 .
drwxr-xr-x 14 root root 4096 Dec 25 03:18 ..
drwxr-xr-x  2 fido fido 4096 Dec 25 03:18 fido
``````

`app.py` - fido application.

``````# /usr/bin/env python

app.config['SQLALCHEMY_DATABASE_URI'] = "postgres://fido:release@localhost:5432/fido"
db = SQLAlchemy(app)

class Todo(db.Model):
id = db.Column(db.Integer(), nullable=False, primary_key=True)
name = db.Column(db.UnicodeText(), nullable=False)
status = db.Column(db.Boolean(), default=False, nullable=True)

@app.route("/")
def index():
return "Index page. Use /new create a new todo"

@app.route('/new', methods=['POST'])
def new():
form = request.form
name, status = form.get('name'), form.get('status') or False
todo = Todo(name=name, status=status)
db.session.commit()
return "Created todo: {}".format(name)

if __name__ == "__main__":
db.create_all()
app.run('0.0.0.0', port=3333, debug=True)
``````

Add a `wsgi` file `website.py`

``````root@fido:/var/www/fido# cat website.py
import sys
import os.path
sys.path.insert(0, os.path.dirname(__file__))
from app import app as application
``````

Files in `fido` directory.

``````root@fido:/var/www/fido# tree .
.
├── app.py
├── __init__.py
└── website.wsgi

0 directories, 3 files
``````

### Step 5 - Virtual env and dependencies

``````root@fido:/var/www/fido# virtualenv --no-site-packages env
root@fido:/var/www/fido# . env/bin/activate
``````

### Step 6 - final setup

Create uwsgi config file

``````# Add following lines to fido.ini file
root@fido:/etc# cat uwsgi/apps-enabled/fido.ini
[uwsgi]
socket = 127.0.0.1:5000
master = true
uid = fido
gid = fido
chdir = /var/www/fido
home = /var/www/fido/env/
pp = ..
module = website
``````

Check whether uwsgi is booting up properly.

``````root@fido:/var/www/fido# uwsgi --ini /etc/uwsgi/apps-enabled/fido.ini
...
...
Python version: 2.7.5+ (default, Sep 19 2013, 13:52:09)  [GCC 4.8.1]
Set PythonHome to /var/www/fido/env/
Python main interpreter initialized at 0xb96a70
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 165920 bytes (162 KB) for 2 cores
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xb96a70 pid:  17559 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 17559)
spawned uWSGI worker 1 (pid: 17562, cores: 2)
``````

`uwsgi` is able to load the app without any issues. Kill the uwsgi using keyboard interrupt.

Now lets create table.

``````root@fido:/var/www/fido# . env/bin/activate
(env)root@fido:/var/www/fido# python app.py
* Running on http://0.0.0.0:3333/
``````

Exit the program, `db.create_all()` must have created the table. Normally in production environment, it is advised to use `python manage.py db create` or any similar approach.

Configure nginx

``````root@fido:/var/www/fido# cat /etc/nginx/sites-enabled/fido.in
server 127.0.0.1:5000;
}

# configuration of the server
server {
# the domain name it will serve for
listen 127.0.0.1; # This is very important to test the server locally
charset     utf-8;

location / {
include uwsgi_params;
}
}
``````

Now `nginx` and `uwsgi` will be running in background. Restart them.

``````root@fido:/var/www/fido# service nginx restart
* Restarting nginx nginx                                                             [ OK ]
root@fido:/var/www/fido# service uwsgi restart
* Restarting app server(s) uwsgi                                                    [ OK ]
``````

Machine name is `fido`, so lets try `curl http://fido`

``````root@fido:/var/www/fido# curl http://fido
Index page. Use /new create a new todo
``````

``````root@fido:/var/www/fido#curl --data "name=write blog post about flask deployment"  http://fido/new
``````

We have successfully deployed `flask` + `uwsgi` + `nginx`.

Since we installed `uwsgi` from ubuntu repo, it is started as `upstart` process, that is why we issue commands like `service uwsgi restart`.

To see all upstart service, try `service --status-all`.

If you are running multiple web application in a single server, create one user per application.

22 12 / 2013

`ipython nbconvert` has lot of handy options to convert `ipynb` to `markdown`, `html` etc… But I wanted to upload `ipynb` to `gist.github.com` and create a link in `nbviewer.ipython.org`. Started with `curl` and soon realized, it is getting messy. So wrote a small python program `ipynb2viewer`, Source code.

## Install

• `pip install ipynb2viewer`

## Usage

Upload all ipynb files in the given path to gist.github.com and return nbviewer urls.

• `ipynb2viewer all <path>`

Upload mentioned file to gist.github.com and return nbviewer url.

• `ipynb2viewer file <filename>`

Upload mentioned file to gist.github.com and open nbviewer url in webbrowser.

• `ipynb2viewer file <filename> --open`

Upload all ipynb files in the given path to gist.github.com and open nbviewer urls in webbrowser.

• `ipynb2viewer all <path> --open`

## Example

``````➜  ipynb2viewer git:(master) ipynb2viewer all tests --open

Uploaded test.ipynb file to gist https://gist.github.com/8081202
nbviewer url: http://nbviewer.ipython.org/gist/anonymous/8081202
Uploaded try.ipynb file to gist https://gist.github.com/8081203
nbviewer url: http://nbviewer.ipython.org/gist/anonymous/8081203
``````

22 11 / 2013

I hardly use `easy_install`. Nowadays all the python requirements are installed via `pip`.

IPython is my primary python console. After installing mavericks, I installed IPython and fired IPython console. Following warning message appeared

``````➜  ~  ipython
/Library/Python/2.7/site-packages/IPython/utils/rlineimpl.py:94: RuntimeWarning:
******************************************************************************
libedit detected - readline will not be well behaved, including but not limited to:
* crashes on tab completion
* corrupting long-lines
* failure to wrap or indent lines properly
It is highly recommended that you install readline, which is easy_installable:
Note that `pip install readline` generally DOES NOT WORK, because
it installs to site-packages, which come *after* lib-dynload in sys.path,
where readline is located.  It must be `easy_install readline`, or to a custom
******************************************************************************
RuntimeWarning)
Python 2.7.5 (default, Aug 25 2013, 00:04:04)
``````

IPython complains `readline` is missing and insisting to use `easy_install`. Then I tried

``````➜   ~  sudo easy_install-2.7 --upgrade setuptools
Traceback (most recent call last):
File "/usr/bin/easy_install-2.7", line 7, in <module>
File "/Library/Python/2.7/site-packages/pkg_resources.py", line 2797, in <module>
parse_requirements(__requires__), Environment()
File "/Library/Python/2.7/site-packages/pkg_resources.py", line 576, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: setuptools==0.6c12dev-r88846
``````

I know I can’t use pip, but `distribute` is already installed.

Now only way to install readline is to use `easy_install` in Python site-packages. Normally `easy_install` is installed in `/usr/bin`.

``````➜   ~  which easy_install
/usr/bin/easy_install
➜  ~  sudo python /Library/Python/2.7/site-packages/easy_install.py readline

➜  ~  ipython
Python 2.7.5 (default, Aug 25 2013, 00:04:04)

IPython 1.1.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
``````

Whenever issue pops up with `easy_install` and `setuptools`, remember using `easy_install` from `site packages` like `sudo python /Library/Python/2.7/site-packages/easy_install.py`

12 10 / 2013

How to check given text is palindrome or not

``````def sanitize(text):
for char in [" ", ".", ",", ";", "\n"]:
text = text.replace(char, "")
return text

def palindrome(word):
# This approach is o(n), problem can be solved with o(n/2)
# I am using this approach for brevity
return word == word[::-1]

palindrome(sanitize(u"விகடகவி")) # False
``````

Here is the hand made version for Tamil

``````# Assign the variable meal the value 44.50 on line 3!
# hex values from க..வ
def sanitize(text):
for char in [" ", ".", ",", ";", "\n"]:
text = text.replace(char, "")
return text

dependent_vowel_range = range(0xbbe, 0xbce)

def palindrome_tamil(text):
front, rear = 0, len(text) - 1
while True:
#We will start checking from both ends
#If code reached centre exit
if front == rear or abs(front - rear) == 1:
return True
else:
if ord(text[front+1]) in dependent_vowel_range and ord(text[rear]) in dependent_vowel_range:
if text[front] == text[rear-1] and text[front+1] == text[rear]:
front += 2
rear -= 2
else:
return False
else:
if text[front] == text[rear]:
front += 1
rear -= 1
else:
return False

print palindrome_tamil(sanitize(u"விகடகவி")) == True
text = u"""
யாமாமாநீ யாமாமா யாழீகாமா காணாகா
காணாகாமா காழீயா மாமாயாநீ மாமாயா
"""
print palindrome_tamil(sanitize(text)) == True

#output
True
True
``````

12 10 / 2013

Tamil has 247 characters. No panic. It is simple. 12 uyir eluthu(அ,ஆ..ஔ) , 18 mei eluthu(க்,ங்..) , 216 uyirmei eluthu(12 * 18 க,ங ).1 ayutham(ஃ).

I assume you know what is unicode. If not read The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets and then read wikipedia page. You will understand most of it. Back to the post.

Every character or letter in unicode has value called `code point`. This is similar to ASCII where value of `a` is 97. All code point value is represented in hexadecimal. Tamil unicode character range starts from `0B80 - 0BFF`. Unicode consortium has a complete mappings. So value of அ is `0B85`.

You don’t believe me ? I will show you the code. Fireup `python2` console(I am using ipython).

``````In [343]: print(unichr(0x0b85))
அ
``````

How to find hex of a character ?

``````In [344]: print(hex(ord(u'க')))
0xb95
``````

What is the code point of கி?

``````In [345]: print(hex(ord(u'கி')))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-345-9e02bf5958df> in <module>()
----> 1 print(hex(ord(u'கி')))

TypeError: ord() expected a character, but string of length 2 found
``````

Python says len of the unicode character `கி` is 2. Lets check.

``````In [346]: print(len(u'கி'))
2

In [348]: print(len(unichr(0x0b85)))
1

In [349]: print(unichr(0x0b85))
அ
``````

So clearly `கி` is composed of two characters.`கி`= `க்`+ `இ` from language point of view. Lets see what program says.

``````In [355]: for c in u'கி':
print c, hex(ord(c))
.....:
க 0xb95
ி 0xbbf
``````

When you are typing in browser or editor, you use `க` + `இ` after that it is the work of font and others work to replace it.

Below is the javascript code executed in chrome console.

``````chr = "கி"
for (var i in chr){console.log(chr[i]);}
க
ி
undefined
``````

So irrespective of the programming language or console or browser unicode is stored in the same way.

`ி`is dependent vowel sign. Let me list all of them.

``````In [362]: for c in range(0xbbe, 0xbce):
print c, unichr(c)
.....:
3006 ா
3007 ி
3008 ீ
3009 ு
3010 ூ
3011 ௃
3012 ௄
3013 ௅
3014 ெ
3015 ே
3016 ை
3017 ௉
3018 ொ
3019 ோ
3020 ௌ
3021 ்
``````

`௅` is a placeholder and that hex value doesn’t have any symbol or character. Dependent vowel will be used for replacing uyir eluthu.

Now lets read character by character in a word .

``````In [366]: for i in u'விகடகவி':
print i, hex(ord(i))
.....:
வ 0xbb5
ி 0xbbf
க 0xb95
ட 0xb9f
க 0xb95
வ 0xbb5
ி 0xbbf
``````

Well above logic holds true for english but not for Tamil.

``````In [395]: word = u'விகடகவி'

In [396]: dependent_vowel_range = range(0xbbe, 0xbce)

In [397]: pos, stop = 0, len(word) - 1

In [398]: while not pos >= stop:
#check if next char is dependent vowel if so, print together
if ord(word[pos+1]) in dependent_vowel_range:
print word[pos], word[pos+1]
pos += 2
else:
print word[pos]
pos += 1
வ ி
க
ட
க
வ ி
``````

Now try to write a program that will find a tamil word is palindrome or not and see the fun.

Note: I need to figure font that will club dependent vowel and print properly in terminal.

Tags:

13 9 / 2013

Tests are mandatory for packages, modules, web apps. If you are lazy to write unit tests but wanted to see the coverage for web app here is the short cut.

Lets consider a Python web app uses Flask, `runserver.py` runs the development web server. Normally server is started by the command `python runserver.py`. Now use `coverage run runserver.py`.

`coverage` python package will run the python program til program exits or user halts. Once web app testing is complete via browser, run `coverage report -m` this will produce big list of lines covered in modules during execution.

One more handy feature is `coverage html`. This produces html file containing python code highlighting executed line and other. All the html files is produced inside `hmtlcov` directory.

Coverage documentation.

## Sample Output

``````➜ hacknight git:(issue-249) ✗ coverage run --source=hacknight runserver.py
/Library/Python/2.7/site-packages/flask_cache/__init__.py:100:   UserWarning: Flask-Cache: CACHE_TYPE is set to null, caching is  effectively disabled.
warnings.warn("Flask-Cache: CACHE_TYPE is set to null, "
* Running on http://0.0.0.0:8100/
/Library/Python/2.7/site-packages/flask_cache/__init__.py:100: UserWarning: Flask-Cache: CACHE_TYPE is set to null, caching is effectively disabled.
warnings.warn("Flask-Cache: CACHE_TYPE is set to null, "
127.0.0.1 - - [13/Sep/2013 23:08:31] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2013 23:08:34] "GET /kracekumar HTTP/1.1" 200 -
127.0.0.1 - - [13/Sep/2013 23:08:36] "GET /kracekumar/jsfoo-hackday HTTP/1.1" 200 -
^C%
➜  hacknight git:(issue-249) ✗ coverage report -m
Name                           Stmts   Miss  Cover   Missing
------------------------------------------------------------
hacknight/__init__                28      0   100%
hacknight/_version                 3      0   100%
hacknight/forms/__init__           0      0   100%
hacknight/forms/comment           12      0   100%
hacknight/forms/event             42      5    88%   58-59, 62-72
hacknight/forms/participant       12      0   100%
hacknight/forms/profile            6      0   100%
hacknight/forms/project            9      0   100%
hacknight/forms/venue             16      0   100%
hacknight/models/__init__         10      0   100%
hacknight/models/_profile         21     21     0%   3-37
hacknight/models/comment          61     18    70%   34-35, 41, 71-72, 78-91, 95, 98
hacknight/models/event           110     43    61%   45, 48-51, 84, 87-88, 91-93, 96-97, 100-109, 112-133
hacknight/models/participant      31      5    84%   34-37, 41
hacknight/models/project          72     31    57%   42-46, 50, 55, 58, 63, 68-85, 101-105
hacknight/models/user             21      5    76%   20, 24, 28, 32, 35
hacknight/models/venue            21      2    90%   26-27
hacknight/models/vote             38     16    58%   17-18, 21-30, 33-36, 39
hacknight/tests/__init__           0      0   100%
hacknight/tests/test_data          3      3     0%   3-7
hacknight/tests/test_models       17     17     0%   4-93
hacknight/views/__init__           8      0   100%
hacknight/views/event            244    192    21%   28-33, 41-56, 70-87, 97-116, 131, 140-145, 155-182, 191-222, 231-260, 269-276, 285-301, 310-313, 325-344, 354-373
hacknight/views/index             36     23    36%   13-15, 20, 25, 30, 35, 40, 45-53, 58-65
hacknight/views/login             28     12    57%   15, 21-22, 28-30, 36-37, 42-45
hacknight/views/profile           36     22    39%   17-23, 30-47
hacknight/views/project          323    256    21%   30-49, 61-73, 85-108, 118, 127-240, 252-259, 270-277, 288-295, 306-315, 327-336, 346-357, 369-378, 387-397, 406-416, 426-440, 451-464, 475-501, 514
hacknight/views/sponsor           45     26    42%   18-30, 42-53, 65-68, 81
hacknight/views/venue             47     29    38%   15-16, 22, 28-38, 45-56, 64-66
hacknight/views/workflow          84     30    64%   42-46, 52, 57, 62, 67, 72, 77, 80, 114-119, 127-128, 136, 144, 152, 160, 168, 176, 182, 188, 194, 200, 206, 209
------------------------------------------------------------
TOTAL                           1410    758    46%
``````